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Abstract 

The  growth  in  the  acquisition  cycle  time  of  large  defense  systems  from  what  was 
planned  (i.e.,  schedule  growth)  creates  several  issues  for  defense  acquisition  managers  and 
policy  makers.  These  issues  include  increased  likelihoods  of  cancellations,  changes  in 
requirements,  and  delays  in  the  fielding  of  improved  combat  capabilities  and  replacements 
for  legacy  systems,  which  have  resulted  in  further  cost  and  schedule  growth.  As  a  result. 
Congress,  the  DoD,  and  the  individual  military  services  implemented  several  major  reforms 
to  address  the  cost  and  schedule  growth  of  weapon  systems. 

This  research  presents  an  empirical  model  of  schedule  growth  to  evaluate  the  impact 
of  acquisition  reform  efforts,  defense  budget  changes,  unexpected  inflation,  and  major 
contingency  operations  (war)  on  schedule  growth  of  major  weapon  systems.  A  fixed-effects 
panel  regression  model  was  utilized  to  describe  the  schedule  performance  (using  earned  value 
data)  of  the  major  weapon  system  programs  managed  by  the  Army,  Air  Force,  and  Navy 
from  1980  to  2002.  This  research  found  that  unexpected  inflation  results  in  increased 
schedule  growth.  In  addition,  the  2000  revision  of  the  DoD  5000  series  accounted  for  a 
reduction  in  schedule  growth.  The  other  examined  acquisition  reforms — the  Packard 
Commission  of  1986  and  the  1993-1996  reform  efforts  [e.g.,  the  Federal  Acquisition 
Streamlining  Act  of  1 994  and  the  Clinger-Cohen  Act  of  1 996] — were  not  correlated  with 
schedule  growth.  This  lack  of  a  relationship  suggests  these  reforms  were  not  fully 
internalized  into  the  Department  of  Defense’s  acquisition  process  and  appear  to  have  not 
been  successful  at  limiting  schedule  growth. 
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AN  ANALYSIS  OF  THE  IMPACT  OF  DEFENSE  ACQUISITION  REFORMS 
AND  EXTERNAL  FACTORS  ON  SCHEDULE  GROWTH  OF 
DEFENSE  WEAPON  SYSTEMS 

I.  Introduction 

General  Issue 

In  a  recent  study  of  26  major  Department  of  Defense  (DoD)  acquisition  programs, 
the  Government  Accountability  Office  (GAO,  2006)  found  that  the  development  costs  of 
these  programs  increased  by  37  percent  and  the  acquisition  cycle  time  increased  by  17 
percent.  This  growth  in  the  cost  and  acquisition  cycle  time  (schedule)  of  large  defense 
systems  creates  several  issues  for  defense  acquisition  managers  and  policy  makers.  Due 
to  limited  budgets,  cost  overruns  lead  to  potential  reductions  in  weapon  system  quantities 
and  capabilities  and  increased  budget  instability  in  other  programs  where  funds  are  taken 
in  order  to  pay  for  the  overruns  (Czelusniak  and  Rogers,  1997:62).  Similar  issues  arise 
due  to  schedule  growth,  including  an  increased  likelihood  of  cancellation  due  to 
escalating  costs  and  schedule  delays  (Pinto  and  Mantel,  1990:273).  Additionally,  a 
longer  development  schedule  allows  more  opportunities  to  change  (and  add  to)  the 
technical  requirements  for  the  weapon  system  (Drezner  and  Smith,  1990:1).  In  the 
circumstances  regarding  a  replacement  weapon  system,  a  growth  in  the  schedule  of  this 
system  also  extends  the  planned  life  of  the  existing  system(s)  assuming  there  is  a  desire 
to  maintain  that  war  fighting  capability  (Reinertsen  et  ah,  2002:8).  As  a  result,  additional 
operations  and  support  costs  are  required  to  maintain  the  existing  system  or  systems  until 
the  replacement  system  is  fielded  (Reinertsen  et  ah,  2002:8). 
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Specific  Issue 

Congress,  the  DoD,  and  the  individual  military  services  implemented  several 
major  corporate  reforms  to  address  the  cost  and  schedule  growth  of  weapon  systems 
(Hanks  et  ah,  2005:xiv).  These  reforms  usually  involved  the  removal  of  time-consuming 
and  costly  requirements  of  the  defense  procurement  process  (Hanks  et  ah,  2005  :xiv). 
These  major  corporate  reforms  paved  the  way  for  and  institutionalized  numerous 
operational  level  actions  to  address  one  or  more  of  the  factors  contributing  to  cost  and 
schedule  growth  (Hanks  et  al,  2005:38).  This  research  focuses  on  analyzing  the  schedule 
growth  aspect. 

Several  researchers  focused  on  identifying  the  numerous  factors  contributing  to 
schedule  growth  that  are  addressed  by  the  operational  level  actions  of  the  acquisition 
reforms.  These  factors  include  technical  risk  (Cashman,  1995;  Rodrigues,  2000), 
requirements  stability  (Dawkins,  1987;  McNutt,  1998)  and  planned  schedule  duration 
(Drezner  and  Smith,  1990;  McNutt,  1998).  Other  researchers  evaluated  the  management 
initiatives  implemented  to  address  these  factors  of  schedule  growth,  including 
prototyping  (Tyson  et  ah,  1992),  incentive  contracts  (Tyson  et  ah,  1992),  and 
evolutionary  acquisition  (GAO,  2006).  Other  studies  suggested  cost  and  schedule  growth 
of  acquisition  programs  can  be  contributed  to  additional  factors  that  are  primarily 
external  to  the  acquisition  process.  These  factors  include  budget  instability  (McNutt, 
1998),  contingency  operations  (McNutt,  1998;  Czelusniak  and  Rogers,  1997),  and 
unexpected  inflation  (Smirnoff,  2006). 
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Research  Objective 


While  researchers  (Christensen  et  ah,  1998;  Holbrook,  2003;  Smirnoff,  2006) 
have  assessed  the  impact  of  acquisition  reforms  on  cost  growth,  there  has  yet  to  be  an 
empirically  rigorous  study  on  the  impact  of  reform  efforts  on  schedule  growth. 
Accordingly,  the  primary  purpose  of  this  research  is  to  evaluate  the  effectiveness  of 
acquisition  reform  efforts  on  limiting  schedule  growth  of  major  DoD  weapon  systems.  In 
order  to  isolate  the  influences  of  reforms,  this  research  addresses  the  extent  to  which 
factors  external  to  the  acquisition  process  are  related  to  schedule  growth,  specifically 
defense  budget  changes  (budget  instability),  unexpected  inflation,  and  major  contingency 
operations. 

Scope  &  Methodology 

To  complete  this  research,  data  was  collected  from  the  Defense  Acquisition 
Executive  Summary  (DAES)  reports,  which  include  programmatic  and  earned  value  data 
for  the  major  acquisition  programs  for  all  of  the  services.  More  specifically,  details 
pertaining  to  the  schedule  information  of  the  various  defense  acquisition  efforts  and 
calculations  of  schedule  growth  were  collected  and  analyzed.  Currently,  program 
initiation  is  marked  by  the  entrance  into  the  system  development  phase  (GAO,  2006:6). 
Consistent  with  previous  research,  these  major  defense  acquisition  reforms  are  modeled 
as  dummy  variables  (c.f ,  Smirnoff,  2006)  reflecting  (and  beginning  in)  the  year  in  which 
the  reforms  in  the  revisions  to  the  DoD  5000  series  of  regulations  reflecting  the  DoD’s 
policy  on  defense  acquisition.  ^  The  impact  of  the  major  defense  acquisition  reforms  and 


*  Additional  information,  including  rationale  regarding  the  operationalizations  utilized  in  this  research,  is 
provided  in  Chapter  III. 
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the  previously  identified  external  factors  on  schedule  growth  are  evaluated  using  panel 
regression  consistent  with  the  method  described  by  Smirnoff  (2006). 
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II.  Literature  Review 


This  chapter  examines  the  literature  regarding  sehedule  growth  of  Department  of 
Defense  (DoD)  weapon  systems.  This  chapter  begins  by  describing  the  prineipal  factors 
contributing  to  schedule  growth.  This  section  includes  an  exploration  of  several 
management  initiatives  implemented  by  the  DoD  to  mitigate  the  risk  and  impact  of  many 
of  these  faetors  on  a  program’s  suecess.  Then,  this  ehapter  describes  several  efforts 
initiated  by  the  DoD  and  Congress  to  reform  weapon  system  aequisition.  This  section 
also  includes  a  review  of  the  researeh  analyzing  the  effectiveness  of  the  reform  efforts. 

Acquisition  Reform 

As  a  result  of  the  eost  and  sehedule  growth  of  weapon  systems,  Congress  and  the 
DoD  introduced  several  reforms  to  improve  the  efficiency  and  responsiveness  of  the 
defense  acquisition  process  (including  the  laws,  regulations,  and  training)  (Hanks  et  ah, 
2005  :xiv).  These  reforms  traditionally  involved  streamlining  requirements,  speeding  up 
processes,  reducing  overhead,  and  eutting  paperwork  in  order  to  reduce  the  bureaucracy 
of  defense  procurement  (Hanks  et  ah,  2005  :xiv).  These  large  scale  eorporate  reforms 
translated  into  numerous  operational  level  aetions  to  address  one  or  more  of  the  factors 
contributing  to  cost  and  schedule  growth  (Hanks  et  al,  2005:38).  The  following  section 
examines  these  factors  and  several  management  initiatives  (i.e.  the  operation  level 
actions)  to  mitigate  the  risk  and  impaet  of  these  factors.  An  exploration  of  key 
aequisition  reform  efforts  follows,  whieh  ineludes  a  diseussion  of  the  reeommendations, 
mandated  ehanges,  and  the  researeh  concerning  the  effectiveness  of  these  corporate 
reform  efforts. 
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Factors  Affecting  Schedule  Growth 

While  not  as  plentiful  as  the  researeh  on  cost  growth,  there  have  been  several 
studies  regarding  the  factors  affecting  schedule  growth.  While  there  are  myriad 
potential  factors  contributing  to  schedule  growth,  it  is  beyond  the  scope  of  this  research 
to  address  all  of  these  factors.^  Nonetheless,  several  key  factors  identified  in  the  existing 
literature  are  discussed,  including  technical  risk,  requirement  changes,  and  planned 
schedule  duration. 

Technical  Risk 

Technical  risk  refers  to  both  the  maturity  of  the  technology  being  incorporated 
into  the  weapon  system  and  the  technical  complexity  of  the  system’s  design  (Cashman, 
1995:73).  The  level  of  technical  risk  increases  when  the  technology  is  less  mature  and 
when  the  design  is  more  complex.  This  greater  technical  risk  often  corresponds  with  a 
greater  probability  and  a  greater  number  of  technical  problems  encountered  in  the  design 
and  development  of  the  components  and  systems  (Cashman,  1995:74).  As  a  result  of  the 
rework  to  address  the  technical  issues,  the  programs  experience  schedule  and  cost  growth 
(Drezner  and  Smith,  1990:23). 

A  study  by  Dawkins  (1987)  conveyed  similar  results  regarding  the  influence  of 
technical  risk  on  schedule  delays.  Studying  delays  on  forty-eight  general  building 
(instructional  facilities,  laboratories,  aircraft  hangars,  office  buildings,  addition  projects, 

^  Most  of  the  factors  affecting  schedule  growth  also  contribute  to  cost  growth,  including  requirements 
changes  (Drezner  et  ah,  1993),  budget  instability  (Singleton,  1991),  technical  risk  (Drezner  et  ah,  1993), 
contingency  operations  (Czelusniak  and  Rogers,  1997),  and  planned  schedule  duration  (Jarvaise  et  ah, 
1996).  This  chapter  alludes  to  this  relationship  on  several  occasions. 

^  See  Drezner  and  Smith  (1990),  Cashman  (1995),  and  Monaco  (2005)  for  an  analysis  of  these  additional 
factors  contributing  to  schedule  growth. 
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and  warehouse  faeilities)  eonstruction  eontraets,  Dawkins  found  teehnieal  difficulty  in 
the  design  as  the  most  significant  factor  for  the  delays  of  the  construction  projects 
(1987:66,69).  The  DoD’s  traditional  strategy  for  acquiring  major  weapon  systems  has 
been  to  plan  programs  that  would  achieve  a  big  leap  forward  in  capability.  This  suggests 
that  there  is  some  inherent  technical  risk  in  most  DoD  weapon  system  programs  because 
the  needed  technologies  often  are  immature,  where  programs  stay  in  development  for 
years  until  the  technologies  are  demonstrated  (GAO,  2003:3).  Rodrigues  (2000:10) 
found  that  those  programs  allowed  to  advance  further  into  system  development  despite 
low  levels  of  technical  maturity  and  unproven  technologies  suffered  the  effects  of  this 
high  technical  risk  with  increased  costs  and  schedule  slips. 

Requirement  Changes 

Requirement  changes  include  increases  or  decreases  in  the  specific  capabilities  of 
the  system  to  be  produced.  These  desired  capabilities  may  change  as  a  result  of 
availability  of  a  newer  technology  or  changing  circumstances,  such  as  programmatic 
difficulties,  availability  of  resources,  and  management  direction.  The  inability  to  meet 
cost,  schedule,  or  technical  performance  requirements  often  forces  the  acquisition 
program  to  suspend  requirements,  increase  funding,  or  to  delay  the  product  (McNutt, 
1998:102).  For  large  DoD  acquisition  projects,  if  these  changes  concern  the  acquisition 
program  baseline  requirements,  the  schedule  delay  is  exacerbated  by  the  requirement  to 
go  through  a  similar  level  of  review  as  the  original  milestone  decision  process  (McNutt, 
1998:102). 

While  not  as  influential  as  technical  difficulty,  Dawkins  found  that  changes  by  the 
owner  did  influence  delays  of  the  construction  projects  significantly  (1987:66,69).  These 
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owner  changes  are  analogous  to  the  requirement  changes  in  DoD  acquisition  programs. 
McNutt  (1998:276)  found  similar  results  regarding  the  relative  impact  of  requirement 
changes  on  schedule  slips.  In  a  survey  of  175  projects,  requirement  changes  ranked  third 
behind  funding  instability  and  technical  problems  as  the  reason  for  a  development 
program  to  slip — accounting  for  approximately  one  quarter  of  the  total  average  project 
slip  of  approximately  two  months  per  year  (McNutt,  1998:276). 

Planned  Schedule  Duration 

A  program  with  a  longer  development  schedule  has  a  greater  probability  of  a 
schedule  delaying  event  occurring,  such  as  budget  changes  (McNutt,  1998),  technical 
problems  (Drezner  and  Smith,  1990),  cost  overruns  (Jarvaise  et  ah,  1996),  and 
requirement  changes  (Drezner  and  Smith,  1990). 

A  study  by  Monaco,  however,  had  conflicting  results  regarding  the  relationship 
between  program  duration  and  schedule  growth  (2005:76).  He  found  the  likelihood  and 
magnitude  of  a  schedule  slip  decreases  as  the  development  duration  increases.  The 
speculated  rationale  was  there  were  greater  opportunities  to  adjust  to  a  problem  in  the 
program  when  the  program  had  a  longer  planned  development  schedule  (Monaco, 
2005:106).  Unlike  the  previous  studies  of  schedule  growth,  Monaco  restricted  the 
analysis  to  more  recent  development  programs  (1990-2003)  that  were  at  least  three  years 
into  their  development  phase. 
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Management  Initiatives 

Several  management  initiatives  were  instituted  to  mitigate  the  risk  and  impact 
these  factors  have  on  program  success,  specifically  in  terms  of  cost  and  schedule.  The 
major  acquisition  reforms  (described  in  greater  detail  below)  institutionalized  many  of 
these  management  initiative  as  a  part  of  their  efforts  to  reduced  cost  and  schedule  growth. 
Many  initiatives  still  remain  in  some  form  from  their  creation;  whereas  others  faded  away 
in  favor  of  different  management  initiatives.  For  example,  after  1965,  the  total  package 
procurement  (TPP)  concept  was  utilized  to  reduce  the  cost  risks  bom  by  the  government 
by  having  contractors  bid  on  the  development,  production,  and  support  work  under  one 
fixed-price  contract.  TPP  was  intended  to  discourage  the  practice  of  contractors 
submitting  low  bids  for  development  (in  a  competition)  and  then  profiting  under  sole- 
source  production  contracts.  During  the  early  1970s,  however,  a  number  of  TPP 
programs  had  large  cost  overmns,  which  left  some  contractors  in  need  of  Government 
assistance  (Tyson  et  al.,  1992:11-2).  Similar  experiences  occurred  regarding  fixed-price 
development  contracts  (Tyson  et  al.,  1992:X-8).  As  a  result,  total  package  procurement 
was  discontinued  in  favor  of  cost-plus-incentive-fee  contracts  that  placed  more  risk  on 
the  government  (Tyson  et  al.,  1992:11-2). 

These  incentive  contracts  were  designed  to  motivate  the  contractor  to  meet 
desired  acquisition  objectives  while  discouraging  inefficiency  and  waste  (FAR  16.401a). 
Incentive  contracts  usually  include  cost  objectives,  but  can  also  include  incentives  for 
technical  or  schedule-related  performance  (FAR  16.401b).  Tyson  found  development 
programs  with  incentive  contracts  experienced  less  schedule  growth  than  those  programs 
without  incentive  contracts  (i.e.  fixed-fee  contracts)  (Tyson  et  al.,  1992: VI-3). 
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Another  manner  in  which  the  government  attempts  to  manage  the  risks  of 
program  is  through  the  use  of  prototyping.  Prototyping  is  designed  to  reduce  technical 
risk  by  building  and  testing  detailed  pieces  of  hardware  prior  to  any  large-scale 
development  (Tyson  et  ah,  1992:V-1).  These  working  models  are  used  to  demonstrate  a 
concept,  a  specific  design,  or  an  operational  objective.  In  order  to  reduce  the  potential  for 
issues  in  development,  potential  design  problems  need  to  be  identified  and  resolved  as 
early  as  possible  (Tyson  et  ah,  1992:V-2).  In  an  analysis  of  tactical  aircraft  and 
munitions  from  the  1960s  to  1989,  Tyson  found  no  statistical  difference  between  the 
development  schedule  growth  of  programs  that  were  prototyped  and  those  that  were  not 
(Tyson  et  ah,  1992:V-3).  Overall,  prototyped  programs  took  somewhat  longer,  but  the 
differences  may  be  due  to  technical  complexity  (Tyson  et  ah,  1992:ES-5). 

Advanced  Concept  Technology  Demonstrations  (ACTD)  and  Advanced 
Technology  Demonstrations  (ATD)  are  prototype  efforts  that  are  instrumental  in  the 
initiatives  to  reduce  the  acquisition  cycle  time  of  weapon  systems.  These  experimental 
tools  generated  a  second  parallel  acquisition  process,  which  give  war  fighters  the 
opportunity  to  assess  prototype  operational  capability  prior  to  formal  program  initiation 
(system  development).  The  objective  was  to  bridge  the  gap  and  improve  the  transition  of 
Science  and  Technology  (S&T)  projects  into  the  formal  acquisition  process  (Vollmecke, 
2004:7).  As  part  of  technology  development,  ACTDs  and  ATDs  aim  to  increase  the  level 
of  technical  maturity  of  a  technology  prior  to  implementation  into  a  weapon  system. 

The  evolutionary  acquisition  approach  relies  upon  the  availability  of  mature 
technologies  to  field  an  operationally  useful  and  supportable  capability  in  as  short  a  time 
as  possible  with  the  intent  to  deliver  additional  capability  in  the  future  (Sylvester  and 
Ferrara;  2003:5-6).  Evolutionary  acquisition  stresses  an  incremental  approach  to 
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development,  whieh  eapitalizes  on  the  best  mature  teehnologies  available  at  a  given  point 
in  time.  Evolutionary  aequisition  takes  advantage  of  eoneurrent  engineering  and  reduees 
aequisition  eyele  time  by  minimizing  teehnical  uneertainty  at  the  start  of  the  development 
program  (Sylvester  and  Ferrara;  2003:8).  The  GAO  identified  this  strategy  of  maturing  a 
new  technology  to  a  high  level  of  technology  readiness  prior  to  inclusion  in  a  product  as  a 
commercial  industry  best  practice  and  a  major  determinant  of  the  success  of  new  product 
launches  (Best-Practices,  1999:3). 

Evolutionary  acquisition  is  a  major  shift  from  the  single-step-to-full-capability 
acquisition  strategy  (grand  design  approach)  used  extensively  for  the  past  30  years 
(Sylvester  and  Ferrara;  2003:8).  The  single-step-to-full-capability  strategy  involves  an 
identified  end  state  in  terms  of  capability  requirements,  which  the  program  would  attain 
in  one  (usually  lengthy)  increment  (Sylvester  and  Ferrara;  2003:8).  Not  surprisingly,  in  a 
2006  study,  the  GAO  contended  the  DoD  was  not  effectively  implementing  the 
evolutionary  approach  by  continuing  to  accept  high  levels  of  technology  risk  for 
programs  entering  into  system  design  and  development  (GAO,  2006:12).  Since  2000, 
thirteen  of  the  eighteen  programs  initiated  under  the  revised  acquisition  policy  received 
approval  to  enter  system  development  with  immature  technologies  (GAO,  2006:14). 

Key  Acquisition  Reform  Efforts 

This  section  discusses  the  following  acquisition  reforms  and  their  potential  impact 
on  schedule  execution:  the  Packard  Commission  (1986),  the  National  Performance 
Review  (1993)  and  Federal  Acquisition  Streamlining  Act  of  1994,  and  the  revision  of  the 
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DoD  5000  series  in  2000."^  These  reforms  serve  as  the  basis  for  this  researeh  into  the 
influence  of  acquisition  reforms  on  schedule  growth.  In  addition,  the  literature  suggests 
the  selected  acquisition  reforms  were  singular  in  their  scope  and  potential  impact  on 
schedule  execution.  While  additional  reforms  have  been  implemented  that  may  have  also 
impacted  schedule  growth,  an  analysis  of  these  reforms  is  beyond  the  scope  of  this 
study.  ^  The  following  section  provides  details  regarding  these  reform  efforts,  including 
the  recommendations  and  initiatives  of  these  reforms,  the  implementation  of  these 
initiatives  and  recommendations,  and  studies  evaluating  the  efficacy  of  these  initiatives. 

Packard  Commission  (1986) 

In  July  1985,  President  Reagan  instituted  the  Blue  Ribbon  Commission  on 
Defense  Management  (also  known  as  the  Packard  Commission)  to  investigate  issues 
pertaining  to  defense  management  and  organization  (Blue  Ribbon  Commission, 

1986:27).  In  order  to  address  the  horror  stories  in  the  media  of  excessive  prices  for  parts, 
test  failures,  and  cost  and  schedule  overruns,  a  task  force  was  created  to  specifically 
address  the  military  acquisition  process  (Munechika,  1997:12).  In  order  to  improve  the 
defense  acquisition  system,  which  included  improving  cost  and  schedule  performance  by 
limiting  overruns  and  reducing  acquisition  cycle  time,  the  task  force  established  a 
formula  for  action  for  the  DoD  to  implement  (Blue  Ribbon  Commission,  1986:15).  The 
commission  recommended  stabilizing  the  programs,  balancing  cost  and  performance, 
using  technology  to  reduce  cost,  streamlining  acquisition  organization  and  procedures. 


Due  to  restrictions  in  the  data  obtained  for  this  analysis  (from  1980-2002),  the  revision  of  the  DoD  5000 
series  in  2003  was  excluded  from  this  list  of  key  acquisition  reforms. 

^  For  additional  information  on  other  acquisition  reform  efforts,  see  Reig  (2000),  Munechika  (1997), 
Ferrara  (1996),  and  Vollmecke  (2004). 
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expanding  the  use  of  eommereial  produets,  inereasing  eompetition,  and  raising  the 
quality  of  aequisition  personnel  (Blue  Ribbon  Commission,  1986:52-71). 

Congress  responded  very  enthusias tie  ally  to  the  Paekard  reeommendations  and,  in 
short  order,  enaeted  the  Defense  Aequisition  Improvement  Aet  of  1986  (Ferrara, 
1996:120).  In  addition,  the  DoD  5000  series  was  updated  in  1987  to  inelude  the  Paekard 
Commission  reeommendations,  ineluding  the  new  streamlined  aequisition  ehain  of 
eommand  (Ferrara,  1996:120). 

A  study  by  MeNutt  (1998)  assessed  the  impaet  of  the  Paekard  Commission’s 
reeommendations  on  sehedule  exeeution.  Despite  implementation  by  the  DoD,  he 
eontended  the  reeommendations  on  sehedule  exeeution  were  not  widely  internalized  and 
appeared  to  have  been  unsueeessful  (MeNutt,  1998:50).  His  assessment  relied  on  a 
visual  analysis  of  the  programmatie  sehedule  data  and  interviews  with  government  and 
eontraetor  personnel  in  the  Pentagon,  program  offiees,  and  eontraetor  faeilities.  A  more 
empirieal  analysis  is  needed  to  eonfirm  this  assessment  of  the  Paekard  Commission’s 
impaet  on  sehedule  performanee. 

Searle  (1997),  Christensen  et  al.  (1999),  and  Smirnoff  (2006)  analyzed  the  impaet 

of  the  Paekard  Commission  reeommendations  on  eost  growth.  The  findings  were 

ineonsistent.  Christensen  et  al.  (1999:257)  found  that  eost  performanee  of  defense 

eontraets  worsened  in  the  period  following  the  eommission.  In  eontrast,  Smirnoff 

(2006:78)  more  reeently  eoneluded  that  eost  performanee  improved  as  a  result  of  the 

Paekard  Commission’s  reeommendations.  These  ineonsistent  results  might  be  attributed 

to  the  methods  that  were  used  to  analyze  the  issue.  Due  to  the  size  and  eomplexity  of  the 

Department  of  Defense,  a  large  aequisition  reform  effort  would  likely  take  some  time  to 

implement  (Reig,  2000:38).  In  order  to  address  this  issue,  one  method  has  been  for  the 
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researcher  to  identify  a  treatment  date  of  when  they  considered  reform  efforts  to  be  fully 
institutionalized  in  the  DoD.  Unfortunately,  the  researcher  cannot  be  certain  that  this 
point  in  time  accurately  reflects  when  a  reform  is  fully  implemented.  Searle  (1997)  and 
Christensen  et  al.  (1999)  used  December  31,  1991  as  a  subjective  treatment  date  for 
separating  the  data  into  pre-  and  post-reform.  That  is,  this  date  represented  their 
assessment  of  when  the  Packard  recommendations  were  fully  institutionalized  in  the 
DoD.  They  utilized  statistical  tests  of  the  difference  between  population  means  of  the 
two  periods  to  assess  the  efficacy  of  the  Packard  Commission  (Searle,  1997:65). 

Even  if  there  were  no  uncertainty  around  a  treatment  date,  the  principal  concern 
with  their  studies  is  the  absence  of  external  (control)  variables,  such  as  changes  in 
defense  budgets,  inflation,  or  war,  might  have  biased  the  results  of  these  studies 
(Smirnoff,  2006:20).  Christensen  et  al.  identified  this  bias  as  a  potential  threat  to  the 
internal  validity  of  their  study  (1999:262).  That  is,  these  acquisition  reform  efforts  might 
have  resulted  in  less  cost  growth,  but  this  relationship  could  be  observed  using  their 
method  due  to  the  influence  of  external,  environmental  factors. 

In  order  to  address  the  shortcomings  of  the  previous  studies,  Smirnoff  (2006) 

utilized  panel  regression  to  evaluate  the  impact  of  acquisition  reform  efforts  and  several 

environmental  variables  on  cost  growth.  The  use  of  panel  regression  allowed  for  a  more 

robust  analysis  of  these  factors  compared  to  the  use  of  subjective  treatment  dates  and 

statistical  tests  of  the  difference  between  two  means,  because  it  showed  the  relative 

importance  of  each  variable  and  its  contribution  to  cost  growth  in  a  dynamic  situation 

(Smirnoff,  2006:5).  Additionally,  Smirnoff  used  the  panel  regression  results  to 

statistically  determine  the  lag  structure  (i.e.  treatment  dates)  for  the  implementation  of  the 

acquisition  reform  efforts  instead  of  developing  a  subjective  treatment  date.  Because  the 
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reform  efforts  address  many  of  the  eommon  factors  of  both  cost  and  schedule  growth,  an 
empirical  evaluation  of  schedule  growth  utilizing  a  method  similar  to  Smirnoff  (2006) 
may  yield  similar  results  after  controlling  for  external  factors.  Specifically,  whether  the 
implementation  of  the  acquisition  reforms  stemming  from  the  Packard  Commission 
reduced  schedule  growth. 

1993-1996  Reform  Efforts 

Initiated  in  March  1993  and  led  by  then-Vice  President  Gore,  the  National 
Performance  Review  (NPR)  performed  a  detailed  look  at  all  government  activities  to  find 
areas  for  improvement  (GAO-NPR,  1999:3).  The  initial  NPR  report  contained  384 
recommendations  spanning  all  aspects  of  government  operations,  including  federal 
procurement.  The  report  identified  the  need  for  the  DoD  to  eliminate  regulatory  burden, 
simplify  procurement,  and  rely  more  on  the  commercial  market  (Rogers  and 
Birmingham,  2004:39). 

The  Federal  Acquisition  Streamlining  Act  of  1994  (FASA)  codified  many  of  the 
NPR’s  recommendations.  The  FASA  was  devised  to  overhaul  the  cumbersome  and 
complex  procurement  system  of  the  federal  government.  To  this  end,  the  act  significantly 
modified  or  eliminated  over  225  existing  statutes  (Holbrook,  2003:17-18).  The 
overarching  themes  of  the  FASA  included  a  preference  for  moving  to  commercial 
contracting  methods,  transitioning  the  procurement  process  to  an  electronic  basis, 
eliminating  non  value-added  requirements,  and  eliminating  paperwork  burdens  in  the 
procurement  cycle  (Cooper,  2002:16).  The  Clinger-Cohen  Act  of  1996  further  advanced 
the  changes  made  by  FASA,  providing  additional  opportunities  for  the  DoD  to  further 


15 


streamline  and  reduee  non-value  added  steps  in  the  aequisition  proeess  (Holbrook, 
2003:19). 

The  National  Performanee  Review  (1993),  the  FAS  A  (1994),  and  Clinger-Cohen 
Aet  (1996)  led  to  a  wave  of  aequisition  management  initiatives,  ineluding  variations  of 
the  previously  mentioned  management  initiatives  (e.g.  evolutionary  aequisition,  program 
stability,  advaneed  eoneept  teehnology  demonstration).  Hanks  et  al.  (2005:38)  identified 
46  sueh  initiatives  that  were  initiated  between  1994  and  1996.  In  their  examination  of 
these  aequisition  management  initiatives.  Hanks  et  al.  grouped  these  reform  initiatives 
into  five  themes: 

-  Rationalizing  and  improving  the  industrial  base 

-  Streamlining 

-  Civilian-military  integration 

-  Logisties  transformation  and  total  life-eyele  system  management 

-  Redueing  fraud,  waste,  and  abuse  (Hanks  et  al.,  2005:122-124) 

More  than  half  of  these  aequisition  management  initiatives  foeused  on  ways  to  streamline 
the  aequisition  proeess,  whieh  often  meant  relaxing  some  of  the  eontrols  established  in 
the  1980s  to  guard  against  fraud,  waste,  and  abuse  (Hanks  et  al.,  2005:16). 

Most  of  these  initiatives  were  instituted  in  the  1996  revision  of  the  DoD  5000 
series.  The  1996  version  refieeted  how  the  aequisition  system  responded  to  the  ehanging 
global  environment  since  the  end  of  the  Cold  War.  Due  to  the  uncertainty  of  where  and 
when  threats  to  the  United  States  could  come,  the  acquisition  system  needed  the 
flexibility  to  be  able  to  respond  very  quickly  (Ferrara,  1996:123).  The  1996  version 
instituted  Advanced  Concept  Technology  Demonstrations  in  order  to  infuse  new 
technology  into  the  process  (Ferrara,  1996:123). 

Integrated  Product  Teams  (IPTs)  were  also  institutionalized  under  this  revision. 

The  purpose  of  IPTs  was  to  breakdown  the  barriers  between  different  organizations  and 
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acquisition  disciplines  and  to  encourage  integrated  solutions  to  management  problems 
(Ferrara,  1996:123).  The  1996  revisions  also  added  the  management  eoneept  of  eost  as 
an  independent  variable,  whieh  involves  making  eost  more  of  a  eonstraint  and  less  of  a 
variable.  As  a  result,  tradeoffs  were  made  between  performanee  requirements  and  life- 
eyele  eosts  in  order  to  meet  eost  objeetives  (Hanks  et  ah,  2005:92).  Additionally,  the 
revision  ineluded  strategies  to  reduee  expensive  testing,  in  terms  of  time  and  money, 
through  the  use  of  modeling  and  simulation.  Finally,  the  1996  revision  signifieantly 
redueed  the  amount  of  paperwork  as  eompared  to  the  previous  (1991)  version  by 
eaneeling  numerous  previously  mandated  report  formats  (Ferrara,  1996:127). 

Similar  to  the  studies  of  the  effeetiveness  of  the  Paekard  Commission’s 
reeommendation,  studies  on  the  impaet  of  the  1993-1996  reform  efforts  yielded 
eonflieting  results.  The  National  Performanee  Review  (NPR)  elaimed  its  efforts  to 
reinvent  the  government  generated  $137  billion  in  savings  between  1993  and  2000 
(GAO-NPR,  1999:1).  A  study  by  the  GAO  eontended  this  number  was  exaggerated, 
beeause  the  NPR  figure  took  eredit  for  all  of  an  ageney’s  savings  (eompared  to  a 
baseline)  over  that  time  period,  even  the  savings  resulting  from  separate  reform  initiatives 
that  were  eonsistent  with  the  NPR  prineiples  of  reinventing  government  (GAO-NPR, 
1999:1-2).  Due  to  the  eomplexity  and  time  required,  the  GAO  did  not  ealeulate  the 
aetual  savings  that  were  solely  attributable  to  the  NPR  reeommendations  (GAO-NPR, 
1999:1).  These  studies  did  not  assess  the  impaet  of  the  NPR  reeommendations  on 
sehedule  growth. 

Smirnoff  found  the  passing  of  the  FASA  eorresponded  with  redueed  eost  growth 
regardless  of  eontraet  type  (2006:73).  Holbrook  eontended  the  FASA  and  the  Clinger- 
Cohen  Aet  did  not  have  an  impaet  on  eost  growth;  however,  he  utilized  a  subjeetive 
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treatment  date  of  Deeember  31,  1997,  for  his  analysis  and  did  not  eontrol  for  the  potential 
influenee  of  any  external  faetors  (2003:91).  As  a  result,  his  study  suffers  from  the  same 
limitations  as  those  found  in  the  studies  by  Searle  (1997)  and  Christensen  et  al.  (1999)  on 
the  impaet  of  the  Paekard  Commission  reeommendations  on  eost  growth. 

As  a  result  of  the  focus  by  researchers  on  cost  growth— specifically  the 
effectiveness  of  DoD  policies  and  reform  initiatives  on  controlling  cost  growth— there  has 
yet  to  be  a  study  assessing  the  impact  of  the  NPR,  FASA,  or  the  Clinger-Cohen  Act  on 
schedule  growth.  Many  of  these  acquisition  reform  initiatives  were  aimed  at  addressing 
the  underlying  causes  of  cost  and  schedule  growth,  such  as  reducing  the  technical  risk 
and  ensuring  requirements  stability,  that  add  unplanned  work  to  the  development  effort. 
As  a  result,  an  empirical  analysis  of  the  impact  of  these  acquisition  reform  efforts  on 
schedule  growth  may  produce  results  consistent  with  the  studies  concerning  cost  growth. 

DoD  5000  Revision  (2000) 

The  2000  revision  codified  many  of  the  initiatives  that  were  developed  under  the 
wave  of  acquisition  reform  efforts  in  the  mid-to-late  1990s  but  not  yet  integrated  into  the 
DoD  5000  series.  One  of  the  more  significant  initiatives,  in  terms  of  its  potential 
influence  (assuming  proper  implementation)  on  schedule  performance,  was  evolutionary 
acquisition.  The  2000  revision  made  evolutionary  acquisition  the  preferred  DoD  strategy 
for  rapid  acquisition  of  mature  technology  for  the  user  (Hawthorne,  2003  :np). 

Due  to  the  relatively  recent  focus  on  evolutionary  acquisition  and  challenges  in 
implementing  the  strategy  (GAO,  2006),  there  has  yet  to  be  an  empirical  evaluation  of  the 
impact  of  evolutionary  approaches  on  DoD  weapon  systems  acquisition.  However,  the 
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success  of  commercial  efforts  using  a  similar  approaeh  suggests  evolutionary  aequisition, 


when  properly  implemented,  may  provide  its  theoretieal  benefits  (GAO,  2006:1).^ 

External  Factors  (Control  Variables) 

In  addition,  the  existing  literature  suggests  several  faetors  that  are  primarily 
external  to  the  defense  aequisition  proeess.  In  order  to  alleviate  the  omitted  variable 
issues  in  the  previous  studies  of  aequisition  reforms  and  eost  growth,  external  faetors  are 
ineluded  as  eontrol  variables  in  this  researeh  of  the  effieaey  of  aequisition  reform  efforts 
to  limit  sehedule  growth.  The  following  seetion  details  the  external  faetors  affeeting 
sehedule  growth  and  the  eorresponding  researeh  of  these  faetors. 

Budget  Instability 

A  program’s  requirements  are  often  modified  as  a  result  of  adjustments  to  the 
program’s  budgets  (eurrent  and  projeeted)  (MeNutt,  1998:188).  The  schedule  of  planned 
work  to  be  performed  on  a  program  assumes  a  eertain  amount  of  funding  aeross 
numerous  years.  When  this  funding  is  altered,  the  program  is  faeed  with  a  deeision  to 
reduee  requirements  or  to  obtain  additional  funding  in  another  (typieally  later)  year  and 
streteh  out  the  program.  Teehnieal  performanee  requirements,  however,  are  less  likely  to 
be  ehanged  than  the  length  of  the  sehedule  (MeNutt,  1998:188).  In  a  1998  survey  of  205 
personnel  at  the  Pentagon  and  various  program  offiees,  113  stated  that  the  sehedule  was 
more  likely  to  ehange  than  performanee  requirements;  whereas,  only  46  stated 


^  On  October  30,  2002,  a  memorandum  issued  by  Paul  Wolfowitz,  the  then-Undersecretary  of  Defense  for 
Acquisition,  Technology  and  Logistics,  cancelled  the  current  DoD  5000  series  of  regulations  (Rogers  and 
Birmingham,  2004:46).  An  interim  guidance  was  put  in  place  until  the  revised  DoD  5000  series  was  issued 
in  2003,  which  is  the  current  version.  Several  of  the  key  initiatives  implemented  in  the  2000  revision 
remained,  specifically  evolutionary  acquisition. 
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performance  requirements  would  more  likely  change — a  statistically  significant 
difference  (McNutt,  1998:287-288).  A  similar  statistically  significant  difference  existed 
in  the  tradeoff  of  cost  and  schedule  where  the  schedule  was  more  likely  to  be  extended 
(McNutt,  1998:288). 

In  addition,  McNutt  (1998:276)  surveyed  government  and  contractor  personnel  at 
the  Pentagon,  program  offices,  and  contractor  facilities  representing  175  projects. 

Results  indicated  that  funding  instability  was  the  primary  reason  for  a  development 
program  to  slip — accounting  for  one  month  of  the  total  average  project  slip  of 
approximately  two  months  per  year  (McNutt,  1998:276).  Although  there  were  some 
limitations  with  McNutt’s  method  (i.e.,  it  was  a  retrospective  look  that  could  be 
influenced  by  errors  in  recall),  the  individuals  that  were  surveyed  were  closer  to  the 
projects  than  outsiders.  Additionally,  the  individuals  involved  in  the  study  were  those 
that  would  be  updating  higher  headquarters  and  Congress  on  the  program’s  status. 

Contingency  Operations 

Existing  research  has  conflicting  results  concerning  the  influence  of  contingency 
operations  on  schedule  growth,  depending  on  the  immediacy  of  the  need  for  the  weapon 
system.  One  side  suggests  the  needs  of  contingency  operations  require  systems, 
especially  those  in  or  near  production,  to  be  fielded  sooner  so  that  the  capability  is 
available  to  the  war  fighters  (McNutt,  1998:40).  As  a  result,  those  weapon  systems 
identified  as  requirements  for  contingency  operations  are  expected  to  have  less  (or 
negative — fielded  prior  to  original  plan)  schedule  growth. 

The  other  side  addresses  those  programs  not  specifically  required  or  hastened  due 
to  contingency  operations.  Czelusniak  and  Rodgers  found  that  Congressional  decisions 
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to  shift  funds  to  near-term  priorities  external  to  these  programs  (e.g.  unplanned 
eontingeney  operations)  aeeounted  for  up  to  one-half  of  the  eost  growth  in  major 
weapons  systems  (1997:59).  Their  analysis,  however,  did  not  extend  into  the  effeets  on 
sehedule  growth.  The  expeetation  is  that  the  funding  needs  related  to  eontingeney 
operations  (ineluding  the  aeeelerated  programs)  eontribute  to  budget  instability  in  the 
programs  not  identified  as  eontingeney  requirements.  This  budget  instability  leads  to 
sehedule  growth  as  previously  diseussed.  Therefore,  the  analysis  of  the  influenee  of 
budget  instability  on  sehedule  growth  would  neeessitate  aeeounting  for  the  instability 
resulting  from  eontingeney  operations. 

Unexpeeted  Inflation 

Cost  estimates  for  major  programs  span  the  entire  life  eyele  of  the  program  from 
initiation  to  disposal,  whieh  ean  last  for  numerous  deeades  into  the  future.  The 
development  phase  of  these  programs  alone  spans  numerous  years.  These  eost  estimates 
ineluded  an  estimate  of  inflation  provided  by  the  OSD  Comptroller.  If  inflation  is 
unexpeetedly  high  in  a  given  year  then  that  foreeasting  error  eould  eontribute  to  eost 
overruns.  There  has  yet  to  be  an  analysis  of  the  impact  of  unexpected  inflation  on 
schedule  growth;  however,  an  analysis  has  been  performed  concerning  cost  growth. 

A  study  by  Smirnoff  (2006:64)  did  not  find  a  statistically  significant  relationship 
between  unexpected  inflation  and  cost  overruns  of  procurement  or  development 
contracts.  Additionally,  the  lack  of  a  relationship  between  unexpected  inflation  and  cost 
overruns  may  be  due  to  the  tradeoffs  of  reduced  performance  requirements  or  extended 
schedules. 
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Summary 

This  chapter  examined  the  principal  factors  affecting  schedule  growth  of 
Department  of  Defense  (DoD)  weapon  systems.  This  ehapter  reviewed  the  major 
aequisition  reforms  with  the  seope  and  potential  to  impact  schedule  exeeution  of  defense 
aequisition  efforts,  which  included  a  review  of  the  research  that  assessed  the  effieacy  of 
these  reforms.  Additionally,  the  literature  identified  funding  instability,  unexpected 
inflation,  and  contingency  operations  as  external  factors  potentially  contributing  to 
schedule  growth.  This  researeh  on  the  efficaey  of  aequisition,  however,  has  primarily 
been  foeused  on  cost  growth.  This  research  provides  the  laeking  empirieal  analysis  of  the 
influence  of  the  aequisition  reform  efforts  and  external  faetors  on  schedule  growth. 
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III.  Data  and  Methods 


This  chapter  details  the  methods  used  in  this  researeh,  including  a  discussion  of 
the  operationalization  of  variables,  the  data  analyzed,  and  the  analytical  techniques 
utilized  in  order  to  address  the  purpose  of  this  research.  More  speeifically,  this  chapter 
discusses  the  empirical  model  that  was  developed,  and  data  that  was  collected,  to 
evaluate  the  effieacy  of  aequisition  reform  efforts  on  limiting  schedule  growth  of  major 
DoD  weapon  systems.  This  model  also  addresses  the  influence  of  factors  external  to  the 
aequisition  proeess  on  schedule  growth,  including  changes  in  defense  budgets, 
unexpeeted  inflation,  and  contingeney  operations. 

The  relationships  between  the  dependent  variable  (sehedule  growth  of  DoD 
weapon  systems)  and  the  independent  variables  (acquisition  reforms,  eontingency 
operations,  unexpected  inflation  rates,  and  budget  changes)  is  evaluated  using  panel 
regression.  In  addition  to  the  discussion  of  the  variables  and  the  corresponding  data,  this 
chapter  discusses  the  advantages  of  the  use  of  panel  regression  in  cross-sectional  time- 
series  analysis. 

Programs 

This  researeh  evaluates  the  sehedule  growth  of  major  defense  aequisition 
programs  (MDAPs)  that  are  categorized  as  aequisition  category  (ACAT)  I  programs. 
These  programs  are  designated  as  ACAT  I  programs  due  to  their  estimated  cost 
exceeding  a  threshold  or  due  to  speeial  interests  (by  the  DoD  or  Congress)  in  the  program 
(DoD5000.2,  2003;np).  This  cost  threshold  is  currently  $355  million  in  fiscal  year  (FY) 
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1996  constant  year  dollars  for  system  development  or  $2,135  billion  in  FY  1996  constant 
dollars  for  production  (DoD5000.2,  2003  :np).  That  is,  if  the  estimated  expenditures  for 
the  program  exeeed  either  of  those  thresholds,  the  aequisition  program  is  eonsidered  an 
MDAP  and  eategorized  by  the  milestone  deeision  authority  as  an  ACAT  I  program. 
Highly  sensitive,  elassified  programs  (as  determined  by  the  Secretary  of  Defense)  are  not 
ineluded  among  the  MDAPs  due  to  their  sensitive  nature  (DoD5000.2,  2003 :np).  These 
programs  reeeive  speeial  designations  and  are  reported  separately.  As  a  result,  this 
researeh  exeludes  elassified  programs  from  the  analysis  and  eoneentrates  on  the  ACAT  I 
major  defense  aequisition  programs.  Eaeh  of  the  military  serviees  is  represented  among 
the  major  defense  aequisition  programs.  Among  these  MDAPs  are  joint  programs,  whieh 
are  designated  by  the  lead  serviee  for  the  joint  program,  such  as  the  Air  Force  being  the 
lead  serviee  for  the  Joint  Direet  Attaek  Munitions  (JDAM)  program. 

As  a  result  of  being  designated  an  ACAT  I  major  defense  aequisition  program, 
these  programs  are  required  to  submit  quarterly  submissions  of  the  program’s  status  into 
the  Defense  Acquisition  Executive  Summary  (DAES)  database.  The  DAES  database 
includes  Earned  Value  Management  (EVM)  data  based  on  information  supplied  to  the 
government  by  the  performing  contractor.  EVM  allows  the  government  and  contractors  to 
monitor  the  status  of  programs  with  regard  to  contract  costs  and  schedule  by  comparing  the 
actual  cost  for  work  performed,  the  budgeted  cost  for  work  performed,  and  the  budgeted  cost 
for  work  scheduled.  Due  to  the  validation  of  the  EVM  accounting  systems  by  the 


24 


government,  the  data  produced  by  these  systems  is  considered  to  be  valid  (Searle,  1997:42). 
The  period  of  analysis  utilized  in  this  research  is  from  1980  to  2002.^ 

This  resulting  sample  DAES  database  is  separated  based  on  service  and  phase  of 
the  program — research  and  development  (R&D)  or  procurement  (proc.).  The  number  of 
entries  corresponding  to  a  specific  contract’s  EVM  data  is  presented  in  the  table  below. 
An  excerpt  from  the  DAES  database,  a  list  of  programs,  and  the  number  of  contract 
entries  per  program  is  provided  in  Appendix  A. 

Table  1:  Number  of  Contract  Entries  in  Sample  DAES  Database 


Air  Force 

Army 

Navy 

Total 

R&D  -  Contract  Entries 

1,998 

1,194 

1,303 

4,495 

Proc.  -  Contract  Entries 

2,172 

792 

2,893 

5,857 

Measures 

Outcome  (Dependent  Variable) 

Schedule  Growth 

Eor  the  purposes  of  this  research,  schedule  growth  is  defined  and  measured  using 
a  modified  version  of  percent  schedule  variance  based  on  the  Earned  Value  Management 

o 

Gold  Card  (2007).  The  following  formula  is  used  to  calculate  these  values  for 
percentage  schedule  growth  of  both  development  and  production  contracts: 


’  This  research  utilizes  the  DAES  database  utilized  in  Smirnoff  (2006),  which  extended  from  1970  to  2002.  This 
database,  however,  did  not  include  data  for  one  of  the  dependent  variables  (unexpected  inflation)  prior  to  1980 
due  to  the  unavailability  of  inflation  estimates.  In  order  to  mitigate  any  potential  confusion  with  a  discussion  of 
varying  beginning  dates,  this  research  focuses  all  discussions/graphical  presentations  on  the  time  period  from 
1980  to  2002. 

*  The  actual  calculation  for  schedule  variance  according  the  EVM  Gold  Card  is  BCWP-BCWS,  which  results  in 
negative  values  reflecting  undesired  circumstances.  In  this  research,  the  variables  were  switched  to  allow  for 
positive  values  to  reflect  percent  schedule  variance  (also  referred  to  as  schedule  growth  throughout  this  study). 
This  adjustment  allows  for  easier  interpretation  and  analysis  of  the  empirical  model  coefficients. 
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%  Schedule  Growth  for  service  i  in  year  t  = 

l^BCWS,,  (1) 

where;  BCWS  =  Budgeted  Cost  for  Work  Seheduled 
BCWP  =  Budgeted  Cost  for  Work  Performed 

This  equation  aggregates  the  values  by  year,  serviee,  and  program  phase  for  all  of  the 
DAES  contracts  entries/years  from  1980-2002.  The  next  chapter  presents  a  visual 
representation  of  the  sehedule  growth  for  both  development  and  production  contracts. 

A  potential  limitation  of  this  operationalization  of  schedule  growth  is  that  it  does 
not  decipher  between  legitimate  delays  eaused  by  DoD  deeisions,  sueh  as  changing 
requirements,  and  delays  caused  by  contraetor  performanee.  The  majority  of  the  delays, 
however,  cannot  be  attributed  to  contractor  actions,  such  as  when  the  DoD  calls  for 
performance  requirements  that  represent  a  high  teehnical  risk  or  when  the  government 
(either  Congress  or  the  DoD)  chooses  to  shift  financial  resources  creating  funding 
instability  (Drezner  and  Smith,  1990:35;  McNutt,  1998:275).  Despite  the  souree  of  the 
delay,  the  issues  resulting  from  schedule  growth  remain,  including  an  increased 
probability  of  eancellation  (Pinto  and  Mantel,  1990:273)  and  costs  associated  with 
maintaining  existing  systems  (Reinertsen  et  ah,  2002:8). 

Antecedents  (Independent  Variables) 

Acquisition  Reform 

The  aequisition  reforms  that  are  studied  in  this  researeh  inelude  the  Packard 
Commission,  the  1993-1996  reforms  (NPR,  FASA,  and  the  Clinger-Cohen  Act),  and  the 
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2000  revision  of  the  DoD  5000  series  of  regulations.  In  aoeordanee  with  previous 
researeh,  these  reforms  are  eoded  as  dummy  variables  with  a  “1”  indieating  the  presenee 
of  the  reform  (Smirnoff,  2006:28-29).  This  researeh  utilizes  the  year  in  whieh  the  DoD 
revised  its  DoD  5000  series  of  regulations  as  the  first  year  of  the  presenee  of  these 
reforms.  A  eommon  thread  throughout  the  seleeted  reforms  is  that  the  DoD  5000  series 
was  revised  soon  after  these  aequisition  reform  efforts  were  eodified  (e.g.,  the  Clinger- 
Cohen  Act)  or  the  study  recommendations  published  (e.g.,  the  Packard  Commission). 
These  revisions  point  to  changes  in  acquisition  policy  made  by  the  DoD  as  a  result  of 
these  major  reform  efforts,  which  would  affect  all  of  the  current  programs  including  the 
major  defense  acquisition  programs. 

In  accordance  with  this  coding  scheme,  the  Packard  Commission 
recommendations  are  coded  as  “I”  from  1987  to  1995.  The  1987  date  reflects  when  the 
DoD  revised  its  acquisition  policy  (DoD  5000  series)  to  incorporate  the 
recommendations.  The  1993-1996  reforms  and  corresponding  management  initiatives 
were  integrated  into  the  1996  revision  of  the  DoD  5000  series;  as  a  result,  these  reforms 
are  coded  as  “1”  from  1996  to  2002.  In  addition,  these  reforms  superseded  many  of 
reform  initiatives/recommendations  of  the  Packard  Commission,  which  is  why  the 
Packard  Commission  ends  at  1995.  Finally,  the  2000  revision  of  the  DoD  5000  series  is 
coded  as  “1”  from  2000  to  2002  (the  last  year  of  analysis  in  this  research). 
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External  Factors 


Budget  Instability 

This  research  utilized  the  actual  procurement  and  development  budgets  as  a  proxy 
measure  for  budget  instability.  Budget  instability  was  measured  by  utilizing  the  constant 
year  budget  figures  detailed  in  Chapter  6  of  the  National  Defense  Budget  Estimates  for 
2006  (also  known  as  the  FY2006  Greenbook).  The  budget  information  in  this  report  was 
collected  for  each  service  (Air  Force,  Army,  Navy),  for  each  appropriation  (research  and 
development  and  procurement),  and  for  each  year  of  the  analysis  (1980-2002).  A  visual 
representation  of  the  resulting  values  is  presented  in  Appendix  B. 

Contingency  Operations 

This  research  codes  contingency  operations  as  a  binary  (dummy)  variable,  in 
which  the  values  of  “1”  correspond  to  the  existence  of  major  combat  operations  (war)  in 
a  given  year  to  account  for  the  influence  of  the  contingency  operations  on  schedule 
growth  of  major  defense  acquisition  programs.  These  major  combat  operations  are  more 
likely  to  impact  major  defense  acquisition  programs,  either  through  budget  instability  as 
suggested  by  Czelusniak  and  Rogers  (1997)  or  through  the  needs  of  combatant 
commanders  hastening  programs  in  order  to  support  the  contingency  operations  as 
suggested  by  McNutt  (1998).  Therefore,  for  the  time  period  of  1980  to  2002,  the 
identified  major  contingencies  (where  the  war  dummy  variable  is  coded  as  “1”)  were 
Desert  Storm/Shield  and  the  Global  War  on  Terror  (Smirnoff,  2006:29). 
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Unexpected  Inflation 

The  OSD  Comptroller  provides  estimates  of  future  inflation  rates  that  the  cost 
estimators  incorporate  into  their  estimates  for  major  defense  programs.  These  estimates 
of  future  inflation  rates  are  presented  in  Chapter  5  of  the  National  Defense  Budget 
Estimates  section  included  in  the  annual  President’s  Budget  submission.  This  research 
measures  unexpected  inflation  as  the  difference  between  actual  inflation  and  expected 
inflation  for  the  year  the  funding  was  used  (Smirnoff,  2006:31).  The  following  formula 
is  used  to  calculate  the  values  for  unexpected  inflation: 

Unexpected  Inflation  =  Inflation^^,^^i  -  Inflation (2) 

For  example,  the  expected  inflation  rate  for  the  DoD  in  1980  was  5.9  percent,  but  the 
actual  inflation  rate  for  1980  was  1 1.7  percent.  As  a  result,  there  was  5.8  percent  of 
unexpected  inflation  for  1980.  This  level  of  unexpected  inflation  was  calculated  for  each 
year  from  1980-2002  to  correspond  with  the  span  of  the  schedule  growth  data.  A  visual 
representation  and  a  table  of  the  calculated  unexpected  inflation  values  are  included  in 
Appendix  B. 

Analysis 

The  influence  of  the  major  defense  acquisition  reforms  and  the  external  factors 
(budget  instability,  contingency  operations  and  requirements,  and  unexpected  inflation) 
on  schedule  growth  is  empirically  evaluated  using  panel  regression.  Panel  regression  is 
utilized  to  analyze  an  independent  variable  across  groups  with  respect  to  multiple  time 
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periods.  In  terms  of  this  researeh,  the  groups  represent  each  service  and  the  time  periods 
correspond  to  the  years  1980-2002. 

Panel  (data)  analysis  provides  several  benefits  when  compared  to  other  possible 
analytical  methods,  such  as  multivariate  regression  analysis  of  cross-sectional  data. 

Panel  regression  provides  the  ability  to  control  for  omitted  variables.  The  bias  resulting 
from  omitted  variables  is  typically  an  issue  with  multivariate  analysis.  This  issue  stems 
from  unknown  variable  or  variables  affecting  the  dependent  variable,  which  leads  to 
biased  estimations  (regression  results)  (Kennedy,  2003:302).  The  issue  of  omitted 
variables  represented  the  principle  limitation  in  the  previous  studies  of  acquisition 
reforms  and  cost  growth  (Christensen  et  ah,  1999;  Holbrook,  2003).  The  ability  of  panel 
regression  to  control  for  these  omitted  variables  is  a  key  benefit  to  this  analytical  method. 
In  addition,  the  combining  of  cross-sectional  and  time-series  data  creates  more 
observations  for  statistical  testing  and  produces  more  variability,  which  makes  the  panel 
model  more  robust  to  multicollinearity  (Kennedy,  2003:302). 

This  research  utilizes  a  fixed-effects  specification  for  the  panel  model  consistent 
with  that  utilized  by  Smirnoff  (2006).  Fixed-effects  panel  regression  assumes  there  are 
nominal  time-series  impacts  on  the  dependent  variable,  but  more  cross-sectional  effects 
(Armstrong,  2006:34).  Smirnoffs  study  similarly  evaluated  the  impact  of  acquisition 
reforms  and  additional  (economic)  variables  on  major  defense  acquisition  programs  using 
a  fixed-effects  panel  model;  however,  his  analysis  focused  on  the  impacts  on  cost  growth. 
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Summary 

This  research  empirically  evaluates  the  influence  of  defense  acquisition  reforms 
and  external  factors  (budget  instability,  contingency  operations,  and  unexpected  inflation) 
on  schedule  growth  of  major  defense  acquisition  programs  between  1980  and  2002.  This 
chapter  described  the  how  the  dependent  variable  (schedule  growth  percentage)  and 
independent  variables  were  operationalized  and  measured,  including  the  corresponding 
rationale.  In  addition,  a  description  of  the  key  source  of  information  pertaining  to  the 
major  defense  acquisition  programs,  the  Defense  Acquisition  Executive  Summary 
(DAES)  reports,  was  presented.  Einally,  this  chapter  described  the  analytical  method  that 
was  utilized  to  analyze  the  relationship  between  the  independent  variables  and  schedule 
growth.  The  results  of  the  panel  analysis  are  presented  in  the  next  chapter. 
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IV.  Analysis  and  Results 


Chapter  Overview 

This  chapter  presents  the  analysis  performed  in  order  to  address  the  purpose  of 
this  researeh.  The  first  section  includes  details  regarding  the  theoretieal  models  utilized 
to  analyze  the  collected  data.  Following  that  section,  the  results  of  these  empirieal 
models  are  presented,  including  a  discussion  of  the  key  findings.  This  discussion 
includes  addressing  the  findings  that  yielded  results  eontrary  to  the  expeeted  (based  on 
previous  researeh)  and  the  variables  that  were  more  significant  (statistically  and  in  the 
magnitude  of  the  coefficients)  than  the  other  variables. 


Theoretical  Model 


The  existing  researeh  suggests  the  below  theoretical  model  for  evaluating  the 
relationship  of  the  previously  discussed  factors  on  schedule  growth: 

'^Budget  Stability,  Unexpected  Inflation,  ^ 


Schedule  Growth  (%)  =  f 


Acquisition  Reforms,  War 


(3) 


The  data  concerning  these  variables  were  colleeted  and  eoded  according  the  methods 
deseribed  in  the  previous  chapter.  As  mentioned  previously,  a  eross-sectional  fixed- 
effects  panel  model  was  utilized  for  this  analysis.  The  general  form  of  this  panel  model, 
where  i  represents  the  service  and  t  represents  the  year,  is:^ 


^  An  alternate  model  speeifieation  ineluded  the  pereent  ehange  in  the  budget  variables.  This  speeifieation, 
however,  ereated  ehallenges  in  the  interpretations  of  the  findings,  speeifieally  in  the  applieations  of  the 
eoeffieients  to  real-world  examples.  As  a  result,  the  aetual  budgetary  figures  were  utilized.  In  addition,  the 
total  budget  for  eaeh  serviee  was  exeluded  due  to  its  high  eorrelation  with  both  the  development  and 
proeurement  budgets,  whieh  are  the  primary  sourees  of  funding  for  major  aequisition  programs. 
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Schedule  Growth  (%);,=  *R&D  Budget j,+/?2*Procurement  Budgetj,+ 

/?3*Unexpected  Inflation j  ,+/?4*Packardi  ,+  /?5*90s  Reforms;,  +  (4) 

/?g*2000  Revision, ,  +(5^  *War; ,  +0, , 


This  panel  model  includes  the  aforementioned  cross-sectional  fixed-effects  to 
account  for  the  service  specific  characteristics  in  the  sample  data.  In  addition,  several 
other  considerations  were  addressed  in  the  modeling  process.  In  order  to  minimize  the 
possibility  for  spurious  correlations,  the  dependent  variable  (percent  schedule  growth) 
was  tested  for  stationarity  using  the  augmented  Dickey-Fuller  test  for  each  panel 
(service).  The  results  of  these  tests  (in  Appendix  C)  suggest  the  existence  of  a  stationary 
process. 

Another  consideration  tested  for  in  the  panel  models  was  autocorrelation,  which 
can  bias  the  regression  results.  Specifically,  it  can  lead  to  an  upward  bias  in  the  estimates 
of  the  statistical  significance  of  coefficient  estimates  (Schmidt,  2003:223).  In  order  to 
address  this  issue,  a  first-order  autocorrelation  component  was  included  in  the  empirical 
models.'' 

An  additional  concern  for  the  models  was  heteroskedasticity  (non-constant 
variance  in  the  residuals),  which  can  also  bias  the  standard  errors  of  the  parameter 
estimates  (Schmidt,  2003:247).  In  order  to  mitigate  the  potential  heteroskedasticity,  the 
residuals  were  ‘White- washed’  using  White’s  heteroskedasticity-consistent  variance- 
covariance  matrix  (White,  1980)  to  obtain  robust  standard  errors. 


Additional  tests  of  stationarity  corroborated  these  results,  specifically  the  Levin,  Lin  and  Chu  test  for  a 
common  unit  root. 

”  The  Durbin-Watson  statistic  test  was  utilized  to  test  for  autocorrelation.  In  all  of  the  models,  the  Durbin- 
Watson  statistic  increased  such  that  the  value  was  closer  to  two  (the  mean  value  for  the  Durbin-Watson  statistic). 
As  a  result,  the  inclusion  of  an  AR(1)  component  decreased  the  uncertainty  that  autocorrelation  is  not  present. 
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Panel  Model  Results 


This  section  presents  and  interprets  the  model  results,  including  a  discussion  of 
differences  between  the  anticipated  and  realized  impacts  (specifically,  the  direction)  of 
each  variable.  Two  models  were  developed  to  determine  the  impact  of  the  acquisition 
reform  efforts  and  the  external  variables  on  schedule  growth — one  for  schedule  growth  of 
development  programs  and  one  for  schedule  growth  of  production  programs. 

Model  1:  Schedule  Growth  of  Development  Contracts 

As  mentioned  in  Chapter  II,  most  of  the  research  concerning  schedule  growth 
addresses  the  growth  in  the  development  phase  of  the  acquisition  process.  The  dependent 
variable  in  this  model  is  the  average  annual  schedule  variance  (in  percentage  terms)  per 
service  for  the  development  programs  in  the  DAES  database  from  1980  to  2002.  This 
variable  is  depicted  in  Figure  1  below.  The  schedule  growth  values  for  the  Navy  have 
two  distinct  peaks  in  the  mid-1980s  and  the  early- 1990s.  Both  the  Army  and  the  Air 
Force  have  peaks  in  the  late- 1980s  (and  1990). 


Schedule  Growth  of  Development  Contracts 


Air  Force  - ■ -  Army 

Navy 


Figure  1:  Schedule  Growth  of  Developmeut  Coutracts  (1980-2002) 
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Table  2:  Regression  Results  for  Schedule  Growth  of  Development  Contracts 


Development  Schedule  Growth  Panel  Model 

Fixed-effects  Regression 

Robust  Standard  Errors 

Group  Varlable(l):  Service 

Variable 

Coefficient 

t-StatlstIc 

C 

-1.039 

-0.962 

R&D  Budget  (CY06  $B) 

0.160 

0.677 

Proc.  Budget  (CY06  $B) 

0.094  t 

1.652 

Unexpected  Inflation 

0.413  *** 

2.794 

Packard 

0.393 

0.453 

90s  Reforms 

-0.370 

-0.310 

2000  Revision 

-1 .048  t 

-1.398 

War 

1.074  t 

1.418 

AR(1) 

0.537  *** 

4.359 

F-statlstIc 

12.118  *** 

Number  of  Obs 

66 

Number  of  Groups 

3 

R-squared 

0.688 

Adjusted  R-squared 

0.631 

Durbln-Watson  stat 

1.670 

***statistical  significance  at  0.01  level, 
**  0.05  level,  *  0.10  level,  f  0.20  level 


The  results  of  the  panel  model  are  presented  in  Table  2  above.  The  first 
variables  of  interest  coneern  the  research  and  development  and  procurement  budgets. 
Changes  in  the  research  and  development  budget  do  not  have  a  statistically  significant 
relationship  with  the  schedule  variances  (used  to  measure  schedule  growth)  of  the 
development  contracts.  On  the  other  hand,  procurement  budgets  have  a  positive, 
statistically  significant  relationship  with  schedule  growth  of  development  contracts.  The 
magnitude  of  this  relationship  (as  measured  by  the  coefficient),  however,  is  not  very 


This  research  first  evaluates  the  contemporaneous  conditions  by  not  including  any  lagged  variables;  however, 
alternative  model  specifications  including  lags  were  also  examined  to  see  the  impacts  of  their  inclusion  in  the 
model.  The  optimal  lag  lengths  determined  by  the  lowest  the  Akaike  Information  Criterion  (AIC)  were  three 
years  for  the  budget  variables,  Packard,  and  the  1990s  reforms.  The  2000  revision  had  a  one  year  lag  and  war 
had  no  lag.  All  of  the  same  variables  as  in  the  original  specification  were  statistically  significant  to  at  least  the 
0.20  level  except  unexpected  inflation  became  no  longer  statistically  significant. 
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large.  The  results  suggest  a  10  billion  dollar  (eonstant  year  2006)  inerease  in  the 
serviee’s  proeurement  budget  would  inerease  sehedule  growth  by  only  about  one  pereent. 
This  considerable  amount  of  budget  growth  represents  an  approximate  sixty  percent 
increase  in  the  procurement  budget  for  the  Army  based  on  the  mean  for  the  sample. 

Unexpected  inflation  has  a  positive,  highly  statistically  significant  relationship 
with  schedule  growth  of  development  contracts.  The  results  state  a  one  percent  value  for 
unexpected  inflation  (such  as  if  inflation  was  estimated  to  be  2.5  percent  but  actually  was 
3.5  percent)  would  increase  schedule  growth  of  development  contracts  by  four-tenths  of  a 
percent,  holding  all  other  variables  constant. 

Two  of  the  four  binary  (dummy)  variables  have  a  statistically  significant 

relationship  with  schedule  growth  in  this  specified  model,  including  major  contingency 

operations  (war)  and  the  2000  revision  of  the  DoD  5000  series  of  regulation  concerning 

the  DoD  acquisition  process.  The  results  indicate  major  contingency  operations  (and  the 

corresponding  large  deployments  and  budget  impacts)  increase  schedule  growth  on 

development  contracts  by  a  little  over  one  percent.  In  addition,  the  2000  revision  resulted 

in  an  approximate  one  percent  reduction  in  schedule  growth.  Both  of  these  variables 

agree  with  the  theorized  impacts  based  on  the  existing  research.  On  the  other  hand,  the 

other  two  variables  representing  the  presence  of  the  reform  efforts  did  not  have 

1  ^ 

statistically  significant  relationships. 


An  alternate  model  specification  utilizing  percent  change  in  the  budget  variables  did  yield  a  slightly 
statistically  significant  (to  the  0.20  level)  for  both  the  90’s  reforms  and  war.  War  has  roughly  the  same 
coefficient  as  in  the  original  specification  (1.16)  and  the  90 ’s  reforms  has  a  fairly  large  negative  coefficient 
of  -2.05  (see  Appendix  E  for  results).  However,  the  budget  variables  and  unexpected  inflation  were  no 
longer  significant  to  even  the  0.20  level  and  the  overall  model  lost  explanatory  power. 
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The  evaluation  of  schedule  variances  in  dollar  terms  is  not  as  useful  as  the 


evaluation  of  cost  variances  (used  often  in  cost  overrun/growth  research).  That  is,  it  does 
not  necessarily  imply  an  impact  to  the  overall  program  schedule  (in  terms  of  completion 
dates  and  changes  in  program  length/duration)  of  the  same  magnitude  as  the  calculated 
schedule  variance. 

Model  2:  Schedule  Growth  of  Production  Contracts 

The  dependent  variable  in  this  model  is  the  average  annual  schedule  variance  (in 
percentage  terms)  per  service  for  the  production  programs  in  the  DAES  database  from 
1980  to  2002.  This  variable  is  depicted  in  Figure  2  below. 


Schedule  Growth  of  Production  Contracts 


-  Air  Force  - ■ -  Army 

-  Navy 


Figure  2:  Schedule  Growth  of  Production  Contracts  (1980-2002) 


The  one  instanee  it  does  is  if  the  sehedule  varianee  refers  only  to  the  efforts  on  the  program’s  eritieal 
path.  The  aetual  impaet  to  the  program’s  sehedule  may  be  either  more  or  less  than  ealeulated  pereentage 
sehedule  growth  depending  on  what  planned  efforts  of  the  program  had  yet  to  be  eompleted. 
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The  values  appear  to  vary  more  for  the  production  contracts  than  the  development 
contracts,  especially  for  the  Army.  This  fluctuation  is  primarily  the  result  of  the  low 
number  of  production  contract  entries  for  the  Army  in  the  DAES  database  (an  average  of 
34  per  year  versus  94  for  the  Air  Force  and  125  for  the  Navy). 

The  results  of  the  panel  model  are  presented  in  Table  3  below.  Unlike  the  model 
for  development  schedule  growth,  none  of  the  budget  variables  have  a  statistically 
significant  relationship  with  production  schedule  growth.  Conversely,  unexpected 
inflation  remains  statistically  significant  in  the  procurement  contracts  model  with  a 
greater  magnitude  positive  relationship  with  schedule  growth  than  in  the  development 


Table  3:  Regression  Results  for  Schedule  Growth  of  Production  Contracts 


Procurement  Schedule  Growth  Panel  Model 

Fixed-effects  Regression 

Robust  Standard  Errors 

Group  Variable(i):  Service 

Variable 

Coefficient 

t-Statistic 

C 

4.386  * 

1.680 

R&D  Budget  (CY06  $B) 

-0.079 

-0.393 

Proc.  Budget  (CY06  $B) 

0.023 

0.423 

Unexpected  Inflation 

0.783  * 

1.807 

Packard 

-0.468 

-0.394 

90s  Reforms 

-1.207 

-0.811 

2000  Revision 

-1.020  t 

-1.327 

War 

0.726 

1.028 

AR(1) 

0.475  *** 

2.890 

F-statistic 

8.348  *** 

Number  of  Obs 

64 

Number  of  Groups 

3 

R-squared 

0.612 

Adjusted  R-squared 

0.538 

Durbin-Watson  stat 

2.212 

***statistical  significance  at  0.01  level, 
**  0.05  level,  *  0.10  level,  f  0.20  level 


The  lower  number  of  entries  for  the  Army  is  likely  the  result  of  having  fewer  major  aequisition  programs 
in  produetion  in  general  and  even  fewer  programs  reeeiving  earned  value  data,  whieh  is  often  not  obtained 
on  fixed-priee  produetion  eontaets. 
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contracts  model.  The  results  indieate  a  one  pereent  level  of  unexpected  inflation  would 
lead  to  a  0.78  pereent  inerease  in  sehedule  varianees  on  produetion  eontraets,  holding  all 
other  variables  eonstant. 

Only  one  of  the  four  binary  (dummy)  variables  has  a  statistieally  significant 
relationship  with  sehedule  growth  in  this  speeified  model,  whieh  is  the  2000  revision  of 
the  DoD  5000  series  of  regulation  eoneeming  the  DoD  aequisition  proeess.  The  2000 
revision  resulted  in  a  one  pereent  reduetion  in  sehedule  growth,  whieh  agrees  with  the 
theorized  impaet  based  on  the  existing  researeh.  On  the  other  hand,  the  other  two 
variables  representing  the  presenee  of  the  reform  efforts  did  not  have  statistically 
significant  relationships.  In  addition,  the  dummy  variable  representing  the  major 
eontingency  operations  (i.e.,  war)  did  not  have  a  statistically  significant  relationship  with 
sehedule  growth  of  proeurement  eontraets. 

Summary 

This  chapter  described  the  analysis  used  to  evaluate  the  impact  of  acquisition  reform 
efforts  and  other,  external  factors  on  schedule  growth  of  major  defense  acquisition  programs 
(MDAP).  The  panel  model  was  first  described  in  general  terms.  Then,  the  regression  results 
of  the  empirical  models  were  presented  and  discussed.  The  models  evaluated  schedule 
growth  for  both  development  and  production  contracts  of  the  MDAPs.  The  next  chapter 
furthers  the  evaluation  of  the  empirical  results  and  their  possible  policy  implications. 


An  alternate  model  specification  akin  to  the  one  described  in  the  development  panel  model  was  also 
generated  for  the  production  contracts.  The  only  variables  to  become  statistically  significant  in  this  revised 
specification  were  the  war  dummy  variable  and  the  percent  change  in  the  development  budget  (significant 
to  the  0.20  level).  Both  unexpected  inflation  and  the  2000  revision  dummy  variable  remained  consistent  in 
both  the  magnitude  of  the  coefficient  and  the  level  of  statistical  significance. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

This  chapter  further  examines  the  results  from  Chapter  IV  through  their 
applieation  to  the  research  objectives  identified  in  the  introduction  (Chapter  I).  That 
section  is  followed  by  a  discussion  of  the  major  findings  of  this  researeh.  Finally, 
reeommendations  for  future  researeh  are  presented. 

Research  Ohjectives 

The  primary  purpose  of  this  research  was  to  perform  an  empirically  rigorous 
study  of  the  impaets  of  aequisition  reform  efforts  and  other,  external  factors  on 
acquisition  schedule  growth  that  was  lacking  in  the  existing  literature.  Entries  from  the 
DAES  database  representing  both  development  and  produetion  eontraets  for  all  three 
services  from  1980  to  2002  were  utilized  to  build  an  empirieal  (panel)  model  in  order  to 
evaluate  the  effectiveness  of  aequisition  reform  efforts  on  limiting  schedule  growth  in  the 
development  of  major  DoD  weapon  systems.  In  order  to  isolate  the  influences  of  the 
reforms,  this  researeh  addressed  the  extent  to  whieh  factors  external  to  the  aequisition 
proeess  are  related  to  sehedule  growth,  speeifically  defense  budget  ehanges  (budget 
instability),  unexpeeted  inflation,  and  eontingency  operations. 

Discussion  of  Results  and  Research  Ohjectives 

This  section  applies  the  results  from  Chapter  IV  to  the  researeh  objectives, 
ineluding  a  discussion  of  potential  conclusions  concerning  the  objeetives  based  on  the 
results  of  the  empirical  models.  The  eombined  results  of  both  panel  models — the 
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development  contraets  model  and  the  production  contracts  model — are  shown  in  Table  4 
below. 


Table  4:  Results  of  the  Panel  Models 


Development  Schedule  Growth  Panel  Model  | 

1  Procurement  Schedule  Growth  Panel  Model 

Variable 

Coefficient 

t-StatlstIc 

Variable 

Coefficient 

t-StatlstIc 

C 

-1.039 

-0.962 

C 

1.680 

R&D  Budget  (CY06  $B) 

0.160 

0.677 

R&D  Budget  (CY06  $B) 

-0.079 

-0.393 

Proc.  Budget  (CY06  $B) 

0.094  t 

1.652 

Proc.  Budget  (CY06  $B) 

0.023 

0.423 

Unexpected  Inflation 

0.413 

2.794 

Unexpected  Inflation 

0.783  ‘ 

1.807 

Packard 

0.393 

0.453 

Packard 

-0.468 

-0.394 

90s  Reforms 

-0.370 

-0.310 

90s  Reforms 

-1.207 

-0.811 

2000  Revision 

-1.048  t 

-1.398 

2000  Revision 

-1.020  t 

-1.327 

War 

1.074  t 

1.418 

War 

0.726 

1.028 

AR(1) 

0.537 

4.359 

AR(1) 

0.475 

2.890 

“‘statistical  significance  at  0.01  level,  **  0.05  level,  *  0.10  level,  t  0.20  level 


Budget  Instability 

This  research  did  not  obtain  results  consistent  with  the  previous  research  that 
suggests  reduction  in  budgets  (a  proxy  for  budget  instability)  would  lead  to  a  schedule 
slip  (and  therefore,  schedule  growth)  (McNutt,  1998:276).  McNutt’s  study  addressed 
schedule  impacts  of  development  programs  identified  through  the  use  of  a  survey 
instrument  (vice  contractual  data)  and  the  budget  stability  referred  to  the  program’s 
development  budget.  In  addition,  his  research  examined  programs  of  varying  sizes  and 
not  just  the  major  weapon  systems  (as  in  this  research).  This  research  did  not  find  that 
changes  in  a  service’s  research  and  development  budget  impacts  schedule  growth  of 
either  procurement  or  development  contracts.  This  lack  of  a  relationship  is  perhaps  the 
result  of  major  defense  development  programs  tending  not  to  have  their  development 
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budgets  dramatically  reduced  unless  the  program  is  terminated. 

Changes  in  the  procurement  budget,  however,  do  appear  to  have  a  statistical 
relationship  with  schedule  growth  for  development  programs.  While  the  literature  does 
not  address  procurement  budget  stability  as  a  source  of  development  schedule  growth,  a 
possible  explanation  may  that  increased  production  budgets  likely  correspond  to  larger 
acquisition  programs  (including  development)  that  are  more  complex  (and  likely  more 
costly).  These  larger,  more  complex  development  programs  are  more  likely  to 
experience  schedule  growth  (Rodrigues,  2000:10).  While  the  changes  in  the  procurement 
budget  may  capture  this  potential  relationship  with  schedule  growth,  a  more  appropriate 
measure  that  is  not  included  in  the  empirical  model  that  may  explain  this  relationship  is 
technical  complexity  (or  technical  maturity). 

Unexpected  Inflation 

While  Smirnoff  (2006)  did  not  find  a  statistical  relationship  between  unexpected 
inflation  and  cost  overruns,  the  postulated  rationale  was  that  the  lack  of  a  relationship 
may  be  due  to  tradeoffs  of  reduced  performance  requirements  or  extended  schedules. 
Based  on  the  results  of  the  empirical  result,  the  latter  appears  to  be  the  case.  This 
research  found  that  unexpected  inflation  impacts  schedule  growth  in  both  development 
and  production  contracts.  That  is,  when  faced  with  a  level  of  inflation  greater  than  the 
forecasted  amount  (a  positive  value  for  unexpected  inflation),  both  development  and 


The  same  eannot  be  said  regarding  produetion  budget  profdes,  whieh  are  often  signifieantly  larger  than 
the  development  budgets  and  ehange  more  dramatieally  as  a  result  of  redueed  quantities  to  proeure  and/or 
due  to  other  (often  politieal)  faetors. 
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production  programs  experience  inereased  sehedule  growth.  For  example,  one  pereent 
level  of  unexpeeted  inflation  would  result  in  a  0.4  pereent  inerease  in  sehedule  growth  for 
development  eontraets  and  0.8  pereent  inerease  for  proeurement  eontraets,  holding  all 
other  variables  eonstant.  This  inerease  in  sehedule  growth  signifies  an  inerease  in  work 
that  was  not  aoeomplished  as  planned.  This  level  of  sehedule  varianee  in  the  eontraets 
likely  eorresponds  to  an  extension  in  the  program’s  sehedules  (sueh  as,  eompletion  and 
milestone  dates)  to  perform  the  missed  work. 

Contingeney  Operations  (War) 

The  existing  researeh  presents  eonflieting  theories  on  the  potential  impaet  of 
major  eontingeney  operations  on  sehedule  growth.  One  side  suggests  eontingeney 
operations  require  systems  to  be  fielded  sooner,  espeeially  those  in  or  near  produetion,  so 
that  the  eapability  is  available  to  the  war  lighting  personnel  (MeNutt,  1998:40).  The 
other  side  suggests  eontingeney  operations  drive  funding  needs  that  eontribute  to  budget 
instability  in  the  programs  not  identified  as  eontingeney  requirements  (Czelusniak  and 
Rogers,  1997:59).  While  the  war  dummy  variable  was  not  statistieally  signifieant  in  the 
proeurement  eontraets  model,  it  was  signifieant  in  the  development  eontraets  model.  In 
this  model,  the  existenee  of  a  major  eontingeney  operation  inereased  sehedule  growth  on 
development  eontraets  by  a  little  over  one  pereent.  Therefore,  the  results  of  the 
produetion  model  agree  with  the  theory  presented  in  Czelusniak  and  Rogers  (1997)  that 
the  budget  instability  generated  by  eontingeney  operations  leads  to  worsened  program 
performanee  in  terms  of  sehedule  growth.  In  addition,  while  a  handful  of  programs  may 
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be  fielded  sooner  than  expected  in  order  to  support  contingency  operations,  the  majority 
of  programs  are  not/* 

Acquisition  Reform  Efforts 

After  accounting  for  the  impacts  of  changes  in  the  defense  budget,  unexpected 
inflation,  and  contingency  operations  on  schedule  growth,  the  results  were  primarily 
inconclusive  regarding  the  impact  of  the  acquisition  reforms  efforts  on  schedule  growth. 
The  existing  research  suggests  the  reforms  utilized  in  the  empirical  model  had  the 
greatest  potential  to  impact  schedule  execution.  These  acquisition  reforms  include  the 
Packard  Commission  (1986),  the  National  Performance  Review  (1993)  and  Federal 
Acquisition  Streamlining  Act  of  1994,  and  the  revision  of  the  DoD  5000  series  in  2000. 

This  research  did  not  find  a  statistical  relationship  between  the  implementation  of 
the  Packard  Commission’s  recommendations  and  initiatives  and  schedule  growth  of 
either  production  or  development  contracts.  Similar  inconclusive  results  were  obtained 
regarding  the  implementation  of  the  1990s  reform  efforts  in  the  1996  revision  of  the  DoD 
5000  series.  The  2000  revision  of  the  DoD  5000  series,  which  included  several  initiatives 
aimed  at  controlling  cost  and  schedule  (specifically,  evolutionary  acquisition),  has  a 


**  This  assertion  is  made  based  on  a  visual  examination  of  the  schedule  milestones  contained  in  the 
Selected  Acquisition  Reports  (SARs)  of  the  major  defense  acquisition  programs  from  1997  to  2005. 
Programs  that  require  quicker  fielding  tend  to  have  this  reduced  duration  from  the  initiation  of  the  program. 
As  a  result,  the  corresponding  reports  would  not  indicate  a  reduction  in  schedule,  because  the  reduced 
schedule  is  the  baseline  for  that  program.  For  example,  the  schedule  for  the  Army’s  Stryker  program  is 
perceived  to  have  been  advanced  due  to  contingency  requirements.  The  first  SAR  reflecting  the  approved 
program  based  on  the  Milestone  II  Review  (in  Nov  2000)  already  has  a  combined  development/production 
(low  rate  initial  production)  decision  with  a  threshold  date  of  January  2003  for  the  first  unit  equipped.  The 
actual  first  unit  equipped  date  was  actually  later  in  March  2003.  If  the  program  had  been  advanced,  it 
would  be  expected  that  this  date  would  be  sooner  than  initially  planned  and  definitely  before  the  threshold 
date.  This,  however,  is  not  the  case. 
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statistically  significant  relationship  with  schedule  growth  of  both  development  and 
proeurement  eontraets.  Furthermore,  this  researeh  found  the  2000  revision  redueed 
development  sehedule  growth  by  1 .05  pereent  and  procurement  schedule  growth  by  1 .02 
pereent.  This  result  agrees  with  the  researeh  suggesting  the  potential  benefits  of  some  of 
the  initiatives  implemented  in  the  2000  revision,  partieularly  evolutionary  aequisition, 
whieh  plaees  greater  emphasis  on  eontrolling  requirements  and  fielding  more  mature 
teehnologies  versus  lengthy  developments  involving  new  and/or  unproven  technologies 
(GAO,  2006:1). 

The  laek  of  a  relationship  with  schedule  growth  for  the  bulk  of  the  reform  efforts 
likely  indieates  these  reforms  were  not  effeetive  at  limiting  scheduling  growth;  however, 
they  also  did  not  result  in  inereased  schedule  growth.  This  ineffieiency  at  limiting 
sehedule  growth  may  be  due  to  the  shortfalls  in  the  polieies  enaeted  as  part  of  these 
reforms,  but  more  likely  are  from  not  fully  implementing  the  reeommendations  and 
initiatives  into  the  DoD  aequisition  proeesses.  MeNutt  eontended  that,  despite 
implementation  by  the  DoD,  the  reeommendations  of  the  Packard  Commission  on 
schedule  exeeution  were  not  widely  internalized  and  appeared  to  have  been  unsueeessful 
(1998:50).  The  results  imply  that  this  might  have  been  the  ease  not  only  for  the  Paekard 
reeommendations  but  also  for  subsequent  reform  efforts. 

Significance  of  Research 

This  study  fills  the  void  in  the  existing  literature  of  an  empirieally  rigorous  study 
of  the  faetors  eontributing  to  sehedule  growth.  In  particular,  several  studies  have 
examined  the  impaets  of  reform  efforts/initiatives  on  cost  overruns  (Christensen  et  ah. 
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1998;  Holbrook,  2003;  Smirnoff,  2006),  but  none  have  yet  to  study  the  impaets  on 
sehedule  growth.  Similar  defieieneies  exist  in  the  literature  eoneerning  the  impaet  of 
unexpeeted  inflation  and  eontingeney  operations  (war)  on  sehedule  growth.  As  a  result, 
this  researeh  ean  form  the  basis  for  future  researeh  involving  empirieal  analyses  of  the 
sehedule  exeeution  of  defense  weapon  system  programs. 

Recommendations  for  Future  Research/Policy  Considerations 

Based  on  the  results  of  this  researeh,  this  seetion  presents  a  few  poliey 
eonsiderations  eoneerning  sehedule  growth  of  major  defense  aequisition  programs.  The 
first  poliey  eonsideration  is  for  greater  resourees  to  be  devoted  to  foreeasting  inflation, 
eonsidering  its  signifieant  impaet  on  sehedule  growth  in  both  development  and 
proeurement  efforts.  The  seeond  poliey  eonsideration  eoneerns  aeeounting  for  the  effeets 
of  eontingeney  operations  (war)  on  development  sehedule  growth  in  the  eosts  for  future 
eontingeney  operations.  These  eosts  may  be  ealeulated  using  the  eost-of-delay  analysis 
method  in  MeNutt  (1998)  that  eonsiders  the  additional  operations  and  sustainment  eosts 
for  the  additional  time  the  system  to  be  replaeed  has  to  remain  in  serviee  and  multiplier 
(or  effioieney)  eosts/savings  that  are  lost  due  to  the  delay  in  the  fielding  of  a  weapon 
system/eapability.  The  final  eonsideration  is  to  plaee  greater  emphasis  on  the 
implementation  of  the  reeommendations  and  initiatives  set  forth  in  the  aequisition 
reforms.  A  2006  GAO  study  supported  this  assessment,  pointing  to  the  issue  of  the  DoD 
allowing  programs  to  enter  into  system  development  with  immature  teehnologies  as  a 
sign  that  the  DoD  had  yet  to  fully  institutionalize  the  revised  (evolutionary)  aequisition 
approaeh  (2006:12). 
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Recommendations  for  future  research  follow  two  general  paths:  performing  a 
similar  analysis  using  a  different  dataset  and  examining  different  factors  to  see  their 
impact  on  schedule  growth.  Future  research  concerning  the  first  group  includes  utilizing 
the  same  method  on  individual  program  data  using  either  earned  value  data  (as  per  this 
study)  in  the  DAES  reports  or  using  schedule  milestones  found  in  the  Selected 
Acquisition  Reports  (SARs).  This  type  of  analysis  allows  for  greater  evaluation  of 
schedule  impacts  based  on  type  of  weapon  system  and  other  programmatic 
characteristics.  In  addition,  the  existing  dataset  can  be  expanded  to  more  recent  years  to 
discern  any  changes  in  the  results  while  allowing  for  an  analysis  of  more  recent 
phenomena,  such  as  changes  in  the  organizational  structure  of  Air  Force  program  offices 
(System  Wings,  Program  Executive  Officer  consolidation/movement). 

Additionally,  other  studies  can  utilize  a  similar  method  to  evaluate  the  impact  of 
other  factors  on  schedule  execution.  The  other  factors  could  include  personnel 
characteristics,  including  education  level,  acquisition  training,  and  the  number  in  a  given 
job  (e.g.,  cost  analysis).  Another  possible  study  concerns  the  evaluation  of  the  common 
assumption  that  schedule  growth  leads  to  cost  growth  or  vice  versa.  Finally,  a  study 
could  be  performed  comparing  the  earned  value  data  in  the  DAES  to  that  of  the  schedule 
milestone  data  (in  month/year  form)  found  in  the  schedule  section  in  either  the  DAES  or 
SAR  reports.  This  study  will  analyze  to  what  extent  changes  in  the  earned  value  data  are 
reflected  in  the  current  estimates  for  completion  of  schedule  milestones. 
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Appendix  A:  Summary  of  DAES  Data 


Example  Entries 

Table  5  below  provides  an  example  of  a  few  entries  in  the  DAES  database.  The 
specific  areas  utilized  in  this  research  were  the  year,  budgeted  cost  for  work  performed 
(BCWP),  and  budgeted  cost  for  work  scheduled  (BCWS).  The  sample  entries  reflect  the 
DAES  entries  corresponding  to  the  May  25,  2002,  DAES  submissions  for  those 
programs. 


Table  5:  DAES  -  Sample  Entries 


SUBMITDATE 

Contract  ID 

Service 

Program  Name 

ACWP 

BCWP 

BCWS 

BAC 

Program 

Phase 

5/25/2002 

F0862698C0027 

Air  Force 

AMRAAM  (AIM-120A) 

128.9 

130.2 

Development 

5/25/2002 

DAAH0199C0121 

Army 

ATACMS  BLK  II 

130.4 

115.6 

123.7 

124.6 

Production 

5/25/2002 

DAAH0199C0154 

Army 

ATACMS  BLK  II 
Cooperative  Engagement 

122.1 

102 

106.3 

132 

Development 

5/25/2002 

N0001997C0069 

Navy 

Capability  (CEC) 
Cooperative  Engagement 

93.3 

95.7 

96.1 

107.9 

Development 

5/25/2002 

N0002499C5110 

Navy 

Capability  (CEC) 

131.1 

132.2 

132.8 

149.9 

Development 

5/25/2002 

N39997993754 

Navy 

DD(X)  Destroyer 

140 

137.4 

141.5 

172.4 

Development 

5/25/2002 

N0002496C2304 

Navy 

DDG  51 

935.8 

850.5 

858.1 

926.7 

Production 

5/25/2002 

N0002496C2305 

Navy 

DDG  51 

1067.7 

968.3 

979.6 

1021.1 

Production 

5/25/2002 

N0002497C5178 

Navy 

DDG  51 

432.8 

448 

452.1 

772.2 

Production 

5/25/2002 

N0002498C2306 

Navy 

DDG  51 

831.2 

747.7 

780.6 

1849.7 

Production 

5/25/2002 

N0002498C2307 

Navy 

DDG  51 

865.9 

851 

912.4 

2599.7 

Production 

5/25/2002 

F0470197C0044 

Air  Force 

GBS 

Globai  Hawk  Unmanned 

186.2 

186.8 

187.3 

187.9 

Development 

5/25/2002 

F3365701C4600 

Air  Force 

aerial  Vehicle 

Global  Hawk  Unmanned 

7.9 

8 

8.2 

83.7 

Development 

5/25/2002 

F3365701C4601 

Air  Force 

aerial  Vehicle 

3.5 

3.5 

3.4 

90.3 

Production 

5/25/2002 

DAAH0198C0033 

Army 

GMLRS  Upgrade  Missile 

125.2 

122.9 

124.9 

135.6 

Development 

5/25/2002 

F0863500C0101 

Air  Force 

JDAM 

Joint  air  to  surface  Standoff 

27.4 

29.5 

29.9 

0 

Development 

5/25/2002 

F0862696C0002 

Air  Force 

Missile  (JASSM) 

Joint  standoff  weapon 

312.5 

295.8 

299.6 

334.3 

Development 

5/25/2002 

N0001995C0120 

Navy 

(JSOW) 

198.2 

190.6 

191.5 

203.2 

Development 

5/25/2002 

F1962800C0023 

Air  Force 

JSTARS 

108.5 

123.1 

122.2 

207.3 

Production 

5/25/2002 

F1962897C0001 

Air  Force 

JSTARS 

293.7 

336.8 

337.4 

358.1 

Production 

5/25/2002 

F1962898C0003 

Air  Force 

JSTARS 

Navistar  Global  Positioning 

143.7 

168.9 

169.3 

202.3 

Production 

5/25/2002 

F0470100C0006 

Air  Force 

system  (GPS)  II  Modern 
Navistar  Global  Positioning 

50.5 

49.8 

49.5 

51.2 

Development 

5/25/2002 

F0470196C0025 

Air  Force 

system  (GPS)  II  Modern 

152 

144 

154 

443 

Development 

5/25/2002 

N0002496C5337 

Navy 

SM  2  (BLKS  l-IV) 

80.4 

78 

78.4 

82.1 

Production 

5/25/2002 

N0001998C0177 

Navy 

Tactical  Tomahawk  Missile 

347.8 

346.6 

346.9 

307.1 

Development 
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Programs 


The  following  tables  list  the  major  defense  aequisition  programs  comprising  the 
dataset  analyzed  in  this  research.  Table  6  lists  the  Army  programs  and  the  number  of 
contract  entries  in  the  overall  DAES  data.  Table  7  lists  the  Air  Force  programs.  Table  8 
lists  the  Navy  programs. 


Table  6:  Army  Programs 


Program  Name 

I  Number  of  Contract  Entries  | 

Research  & 
Development 

Procurement 

ABRAMS  Tank  M1/M1A1 

11 

35 

ADDS 

27 

41 

Advanced  Threat  Infrared  Countermeasures 

25 

0 

AFATDS  (ATCCS) 

49 

0 

AH-64  Apache 

25 

97 

Army  TACMS 

65 

0 

ASAS  (ATCCS)  Block  MB  III 

32 

17 

ATACMS  BLK  II 

94 

10 

BFVS  A3  Upgrade 

29 

0 

BFVS  M2  M3  (Bradley  Fighting  Vehicie 

4 

47 

CH-47  improved  Cargo  Heiicopter  (CH-47F) 

13 

0 

CH-47D  Chinook 

1 

0 

Comanche  Reconnaissance  Attack  Heiicopter  (RAH-66) 

37 

0 

COPPERHEAD 

0 

5 

CSSCS 

0 

21 

FAAD  C2i 

61 

0 

FAAD  NLOS  Elder  Optic  Guided-Misslle 

7 

0 

FBCB2 

0 

3 

GMLRS  Upgrade  Missile 

13 

0 

lAV  (Stryker) 

6 

0 

Javelin 

30 

24 

JSTARS  Common  Ground  Station  (CGS) 

26 

0 

Laser  Hellfire 

28 

58 

Longbow  Apache  FCR 

73 

0 

Longbow  Hellfire 

0 

21 

M1A2  Abrams  Upgrade 

10 

0 

MCS  IV 

23 

0 

MLRS 

45 

4 

MLRS-TGW 

40 

0 

PATRIOT 

72 

120 

Patriot  PAC-3 

91 

9 

PERSHING  II 

21 

69 

ROLAND 

0 

20 

RPV  (AQUILA) 

68 

0 

SADARM 

71 

16 

SCAMP 

10 

0 

SGT  YORK  GUN  (DIVAD) 

0 

31 

SINCGARS 

36 

0 

STINGER 

29 

56 

STINGER  RMP 

0 

56 

TACIT  RAINBOW  (JGL) 

3 

0 

TOW  2 

19 

0 

UH-60A/L  Black  Hawk 

0 

32 

As  mentioned  in  Chapter  III,  the  earned  value  data  detailed  in  the  DAES  entries  for  these  programs  were 
accumulated  for  the  year  and  added  with  the  other  programs  for  that  service  in  that  specific  year. 
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Table  7:  Air  Force  Programs 


Program  Name 

Number  of  Contract  Entries  | 

Research  & 
Development 

Procurement 

ACM 

35 

31 

AFATDS 

2 

0 

ALCM 

8 

24 

AMRAAM  (AIM-1 20A) 

92 

83 

ASAT 

66 

10 

ATS 

20 

0 

B-1  CMUP-DSUP 

2 

0 

B-1B 

110 

255 

B-1BCMUP 

58 

0 

B-2A 

5 

10 

C-130  Avionics  Modernization  Program  ((C-130  AMP) 

2 

0 

C-17A 

128 

99 

CSRL 

14 

9 

□MSP 

40 

136 

DSCS  III  A&B 

21 

15 

DSP 

87 

207 

E-3  AWACS  RSIP 

59 

0 

E-3A 

0 

33 

EF-111A 

0 

9 

EJS 

13 

0 

F-22A  Raptor 

91 

0 

F-15 

15 

62 

F-16 

37 

130 

GBS 

17 

0 

GLCM 

12 

11 

Global  Hawk  Unmanned  aerial  Vehicle 

1 

1 

Inertial  Upper  Stage 

0 

14 

IR  Maverick 

9 

34 

l-S/A  AMPE 

13 

0 

JDAM 

30 

0 

JGL  Tacit  Rainbow 

27 

0 

Joint  air  to  surface  Standoff  Missile  (JASSM) 

16 

0 

Joint  Primary  training  aircraft  (JPATS)  (T-45) 

24 

64 

Joint  Tactical  Information  Distribution  System 

25 

0 

JSIPS  CIGSS 

9 

0 

JSTARS 

79 

102 

KC-135R 

0 

42 

MARK  XV  IFF 

33 

0 

MILSTAR 

32 

17 

Minuteman  III  Guidance  replacement  Program  (MMIII 

38 

21 

Minuteman  III  Propulsion  replacement  program  (MMII 

22 

31 

Navistar  Global  Positioning  system  (GPS)  II  Modern 

84 

98 

OTH-B  (Radar) 

0 

52 

Peacekeeper 

198 

494 

PLSS 

17 

0 

Rail  Garrison 

48 

0 

Sensor  Fused  Weapon 

28 

50 

Small  ICBM 

234 

0 

SMART-T 

20 

0 

Space  based  infra  red  surveillance  system  (SBIRS) 

25 

0 

SRAMTAGM  131A/B 

17 

0 

T-46A 

24 

10 

Titan  IV 

11 

18 

50 


Table  8:  Navy  Programs 


Program  Name 

Number  of  Contract  Entries  | 

Research  & 
Development 

Procurement 

5-INCH  GUIDED  PROJECTILE 

9 

0 

A-12 

9 

0 

Alm-9X  Short  range  air  to  air  missile 

21 

0 

AN/BSY-1 

63 

0 

AN/BSY-2 

26 

0 

AN/SQQ-89 

53 

87 

AN-APG-79  Active  Electronically  Scanned  Array  Radar 

2 

0 

AOE  6 

0 

73 

ASPJ  (AN/ALQ-165) 

0 

33 

AV-8B  Harrier  II 

10 

7 

C/MH-53E 

7 

0 

CG  47  Aegis  Cruiser 

0 

153 

Cooperative  Engagement  Capability  (CEC) 

39 

14 

CVN  68 

0 

66 

DD(X)  Destroyer 

3 

0 

DDG  51 

28 

433 

E-2C  Computer  Upgrade 

63 

0 

EMSP 

12 

0 

F/A-18  C/D 

21 

37 

F/A-18  E/F  Super  Hornet 

69 

60 

F-14D 

0 

6 

FDS 

60 

0 

FFG-7 

0 

28 

HARM  (NAVY) 

27 

5 

Joint  standoff  weapon  (JSOW) 

52 

9 

JTIDS  (NAVY) 

15 

0 

LAMPS  MKIII 

21 

5 

LCAC 

0 

155 

LHD-1 

0 

151 

LPD-17 

0 

45 

LSD  41  CARGO  VAR 

0 

26 

LSD  41  Class  CV 

0 

24 

MCM  1 

0 

31 

MH-60R 

62 

6 

MH-60S 

9 

0 

MHC  51 

0 

112 

MIDS-LVT 

43 

0 

MK  48  ADCAP 

0 

46 

MK  50  Torpedo 

38 

33 

NATO  PHM 

1 

11 

Navy  Area  TMBD 

66 

4 

NSSN  New  Attack  Sub 

63 

104 

P-7A 

6 

0 

PHALANX  CIWS  (MK-15) 

0 

6 

Phoenix  (AIM-54C) 

0 

16 

ROTHR 

3 

0 

SEA  LANCE 

26 

0 

SEALIFT 

0 

123 

SIDEWINDER  (AIM-9L)  (Navy) 

2 

0 

SIDEWINDER  (AIM-9M)  (Navy) 

1 

0 

SM  2  (BLKS  l-IV) 

29 

69 

SPARROW  (AIM-7M)  (Navy) 

0 

14 

SSN  688  Attack  Sub 

35 

203 

T-45TS 

16 

5 

TACTAS 

1 

1 

Tactical  Tomahawk  Missile 

14 

0 

T-AGOS 

0 

20 

T-AO  187  OILER 

0 

26 

Tomahawk  R/UGM-109 

65 

114 

TRIDENT  II  MSL 

79 

187 

TRIDENT  II  SUB 

0 

154 

TRIDENT  SUB 

0 

19 

USMC  H-1  Upgrades 

17 

0 

V-22  Joint  services  advanced  vertical  lift  aircraft 

117 

88 

Virginia  Class  Sub  SSN  774 

0 

84 
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Appendix  B:  Summary  of  Independent  Variables  and  Summary  Statistics 


Procurement  and  Development  Budgets 

Figures  3  and  4  below  show  the  procurement  and  development  budgets  for  the  Anny, 
Air  Force,  and  Navy  between  1980  and  2002.  These  budget  figures  reflect  the  Total 
Obligation  Authority  values  detailed  in  Chapter  6  of  the  National  Defense  Budget  Estimates 
for  2006  (also  known  as  the  FY2006  Greenbook).  The  values  presented  throughout  this  study 
are  in  billions  of  [constant]  fiscal  year  2006  dollars. 


-  Air  Force  - ■ -  Army 

♦  Navy 


Figure  3:  Procurement  Budgets  from  1980-2002  (CY06  SB) 
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Air  Force  - ■ - Army 

Navy 


Figure  4:  Research  &  Development  Budgets  from  1980-2002  (CY06  $B) 


Unexpected  Inflation 

The  estimates  of  future  inflation  rates  are  presented  in  Chapter  5  of  the  National 
Defense  Budget  Estimates  section  included  in  the  annual  President’s  Budget  submission. 
Figure  5  depicts  the  levels  of  expected  inflation  and  actual  inflation  for  each  year  from  1980 
to  2002.  Table  9  provides  the  calculated  values  for  unexpected  inflation  for  that  time  period. 


Figure  5:  Expected  and  Actual  Inflation  Rates  from  1980-2002 
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Table  9:  Unexpected  Inflation  from  1980-2002 


Year 

Expected 
Inflation  (DoD) 

Actual 

Inflation  (DoD) 

Unexpected 

Inflation 

1980 

5.9% 

11.7% 

5.8% 

1981 

8.1% 

10.4% 

2.3% 

1982 

8.9% 

7.5% 

-1.4% 

1983 

6.3% 

3.6% 

-2.7% 

1984 

3.7% 

3.0% 

-0.7% 

1985 

4.5% 

3.3% 

-1.2% 

1986 

4.0% 

2.6% 

-1.4% 

1987 

3.4% 

2.9% 

-0.5% 

1988 

4.5% 

3.6% 

-0.9% 

1989 

3.4% 

3.9% 

0.5% 

1990 

3.0% 

3.0% 

0.0% 

1991 

4.0% 

4.6% 

0.6% 

1992 

2.9% 

1.9% 

-1.0% 

1993 

3.7% 

2.9% 

-0.8% 

1994 

2.0% 

2.3% 

0.3% 

1995 

2.2% 

2.0% 

-0.2% 

1996 

2.8% 

2.2% 

-0.6% 

1997 

2.6% 

2.2% 

-0.4% 

1998 

2.2% 

2.3% 

0.1% 

1999 

2.0% 

2.2% 

0.2% 

2000 

2.2% 

2.5% 

0.3% 

2001 

2.8% 

3.0% 

0.2% 

2002 

3.0% 

2.7% 

-0.3% 

Summary  Statistics 

Table  10  below  provides  the  summary  statistics  for  the  continuous  variables 
utilized  in  this  research.  The  values  reflect  the  entire  range  of  data  from  1980  to  2002  for 
all  three  services.  The  amount  of  schedule  growth  appears  to  vary  more  in  procurement 
contracts,  including  values  reflecting  periods  where  the  programs  of  one  service  were 
ahead  of  schedule  on  average  for  that  year  (that  is,  a  negative  schedule  growth  value). 


Table  10:  Summary  Statistics 


Obs. 

Mean 

Median 

Maximnm 

Minimnm 

Std.  Dev.  1 

R&D  Schedule  Growth  (percent) 

69 

3.25 

2.91 

9.84 

0.47 

Proc.  Schedule  Growth  (percent) 

68 

3.76 

2.60 

17.94 

-2.01 

3.86 

R&D  Budget  ($  billion) 

69 

11.70 

10.35 

22.61 

5.89 

4.60 

Procurement  Budget  ($  billion) 

69 

28.09 

25.64 

61.31 

7.90 

14.06 

Unexpected  Inflation  (percent) 

69 

-0.08 

-0.30 

5.80 

-2.70 

1.57 
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Appendix  C:  Tests  for  Stationarity  of  Dependent  Variable 


In  order  to  minimize  the  possibilities  for  spurious  eorrelations,  the  dependent 
variable  used  in  the  panel  regression  model  must  be  stationary.  In  this  researeh,  the 
dependent  variables  are  the  schedule  growth  percentages  of  procurement  and 
development  contracts  in  the  DAES  database  for  each  service  in  each  year.  The 
Augmented  Dickey-Fuller  Test  was  utilized  in  this  study  to  test  for  the  presence  of  a  unit 
root;  the  values  are  Tables  1 1  and  12  below.  All  of  the  p-values  for  the  tests  are  less  than 
a=0.05,  except  for  that  of  the  Air  Force’s  research  and  development  contracts,  which  has 
a  p-value  less  than  a=0. 15.  This  value  might  have  impacted  some  of  the  findings  in  this 
research;  however,  the  value  is  still  relatively  low  and  the  other  two  services  had  lower 
(more  ideal)  p-values.  This  is  confirmed  in  the  Fevin,  Fin  and  Chu  test  for  common  unit 
root  test  (values  in  Table  13)  with  both  the  procurement  and  development  contract  entries 
having  p-values  less  than  a=0.05.  As  a  result,  the  null  hypothesis  (of  a  unit  root)  can  be 
rejected  and  it  can  be  concluded  that  the  schedule  growth  data  reflect  a  stationary 
process. 


Table  11:  Augmented  Dickey-Fuller  Test  Results  for  Development  Contracts 


Percent  Schedule  Growth  -  Research  &  Development  Contracts 

Dickey-Fuller  Test  for  Unit  Root 


Z(t)  has  t-distribution 


Number  of 

Test 

1%  Critical 

5%  Critical 

10%  Critical 

Obs 

Statistic 

Value 

Value 

Value 

Z(t)-Air  Force  22 

-1.076 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.1474 

Z(t)-Army  22 

-2.877 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.0047 

Z(t)-Navy  22 

-1.81 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.0427 
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Table  12:  Augmented  Dickey-Fuller  Test  Results  for  Procurement  Contracts 


Percent  Schedule  Growth  -  Procurement  Contracts 

Dickey-Fuller  Test  for  Unit  Root 


Z(t)  has  t-distribution 


Number  of 

Test 

1%  Critical 

5%  Critical 

10%  Critical 

Obs 

Statistic 

Value 

Value 

Value 

Z(t)-Air  Force  22 

-2.233 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.0186 

Z(t)-Army  20 

-3.147 

-2.552 

-1.734 

-1.33 

p-value  forZ(t)  = 

0.0028 

Z(t)-Navy  22 

-2.482 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.011 

Table  13:  Common  Unit  Root  Test  Results  for  Schedule  Growth 


Levin,  Lin  &  Chu  Test  for  Common  Unit  Root 

Test 

Number  of  Obs 

Statistic 

Prob. 

R&D  Contracts  64 

-1.68588 

0.0459 

Proc.  Contracts  58 

-3.44782 

0.0003 
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Appendix  D:  Tests  for  Stationarity  of  Continuous  Independent  Variables 


In  addition  to  testing  for  stationarity  of  the  dependent  variable,  unit  root  tests  (same 
as  in  Appendix  C)  were  also  performed  on  the  continuous  independent  variables  (budgets  and 
unexpected  inflation)  utilized  in  this  research.  The  results  of  the  individual  Augmented 
Dickey-Fuller  Tests  are  provided  in  Tables  14-16.  Unexpected  inflation  has  very  significant 
test  statistics  for  both  the  individual  and  common  unit  root  tests,  indicating  a  stationary 
process.  In  addition,  the  research  and  development  budgets  appear  to  reflect  a  stationary 
process  (p-values  less  that  a=0.10  for  the  individual  unit  root  tests  and  around  a=0.10  for  the 
common  unit  root  test).  The  procurement  budgets  appear  to  have  a  greater  possibility  for  a 
non-stationary  process  with  p-values  around  a=0.20.  The  transformation  of  the  budgets  into 
percentage  change  from  previous  year  seems  to  correct  any  issues  with  non-stationarity  in 
both  the  procurement  and  development  budgets.  That  is,  the  p-values  for  both  the  individual 
and  common  unit  root  tests  are  less  than  a=0.01.  The  results  of  these  tests  for  the 
transformed  budget  variables  are  provided  in  Tables  18-20.  In  accordance  with  this 
adjustment,  regression  results  utilizing  these  transformed  budget  variables  are  provided  in 
Appendix  E. 


Table  14:  Augmented  Dickey-Fuller  Test  Results  for  Development  Budgets 


Research  &  Development  Budget  (CY06  $B) 

Dickey-Fuller  Test  for  Unit  Root 

_ Z(t)  has  t-distribution 


Number  of  Test  1%  Critical  5%  Critical  10%  Critical 
Obs  Statistic  Value  Value  Value 


Z(t)-Air  Force 

22 

-2.371 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.0139 

Z(t)-Army 

22 

-1.651 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.0572 

Z(t)-Navy 

22 

-1.497 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.075 
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Table  15:  Augmented  Dickey-Fuller  Test  Results  for  Procurement  Budgets 


Procurement  Budget  (CY06  $B) 

Dickey-Fuller  Test  for  Unit  Root 

Number  of 

Obs 

Test 

Statistic 

Z(t)  has  t-distribution 

1%  Critical 

Value 

5%  Critical 

Value 

10%  Critical 

Value 

Z(t)-Air  Force  22 

-0.817 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.2119 

Z(t)-Army  22 

-0.754 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.2299 

Z(t)-Navy  22 

-0.804 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.2153 

Table  16:  Augmented  Dickey-Fuller  Test  Results  for  Unexpected  Inflation 


Unexpected  Inflation  (%) 

Dickey-Fuller  Test  for  Unit  Root 


Z(t)  has  t-distribution 


Number  of 

Test 

1%  Critical 

5%  Critical 

10%  Critical 

Obs 

Statistic 

Value 

Value 

Value 

Z(t)-Air  Force  22 

-6.01 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0 

Z(t)-Army  22 

-6.01 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0 

Z(t)-Navy  22 

-6.01 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

Table  17:  Common  Unit  Root  Test  Results  for  Budgets  &  Unexpected  Inflation 


1  Levin,  Lin  &  Chu  Test  for  Common  Unit  Root  I 

Test 

Statistic 

Prob. 

Development  Budget  (CY06  $B) 

-1.248 

0.106 

Procurement  Budget  (CY06  $B) 

-0.762 

0.223 

Unexpected  Inflation  (%) 

-7.315 

0.000 
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Table  18:  Augmented  Dickey-Fuller  Test  Results  for  Percent  Change  of 

Development  Budgets 


Percent  Change  -  Research  &  Development  Budget 

Dickey-Fuller  Test  for  Unit  Root 

_ Z(t)  has  t-distribution 


Number  of  Test  1%  Critical  5%  Critical  10%  Critical 
Obs  Statistic  Value  Value  Value 


Z(t)-Air  Force 

22 

-2.664 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.0075 

Z(t)-Army 

22 

-3.277 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.0019 

Z(t)-Navy 

22 

-3.059 

-2.528 

-1.725 

-1.325 

p-value  forZ(t)  = 

0.0031 

Table  19:  Augmented  Dickey-Fuller  Test  Results  for  Percent  Change  of 

Procurement  Budgets 


1  Percent  Change  -  Procuremnt  Budget  | 

Dickey-Fuller  Test  for  Unit  Root 

Number  of 

Obs 

Test 

Statistic 

Z(t)  has  t-distribution 

1%  Critical 

Value 

5%  Critical 

Value 

10%  Critical 

Value 

Z(t)-Air  Force  22 

-2.589 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.0088 

Z(t)-Army  22 

-2.72 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.0066 

Z(t)-Navy  22 

-3.141 

-2.528 

-1.725 

-1.325 

p-value  for  Z(t)  = 

0.0026 

Table  20:  Common  Unit  Root  Test  Results  for  Percent  Change  of  Budgets 


Levin,  Lin  &  Chu  Test  for  Common  Unit  Root 

Test 

Statistic 

Prob. 

%  Change  -  R&D  Budget 

-3.874 

%  Change  -  Proc.  Budget 

-2.769 
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Appendix  E:  Alternative  Specification  for  Panel  Model 


This  section  provides  regression  results  for  the  schedule  growth  panel  models 
utilizing  a  slightly  altered  specification  for  the  models,  which  can  be  found  in  Table  21.  That 
is,  percent  change  of  the  budget  variables  were  used  in  place  of  the  constant  year  2006  dollar 
figures  utilized  previously  (in  the  body  of  this  study).  This  altered  specification  results  in 
fewer  statistically  significant  variables  in  the  model  for  schedule  growth  of  development 
contracts.  In  addition,  the  development  model  explains  less  of  the  variation  in  the  schedule 
growth  data  (an  adjusted  r-squared  value  of  0.568  versus  0.63 1).  On  the  other  hand,  the 
procurement  model  has  more  statistically  significant  variables  and  the  revised  model  explains 
more  of  the  variation  in  the  schedule  growth  data  (from  0.538  to  0.555). 


Table  21:  Alternative  Model  Specification  Regression  Results 


Schedule  Growth  Panel  Models 

Development 

Procurement 

Variable 

Coefficient 

t-Statistic 

Coefficient 

t-Statistic 

C 

4.436  *** 

3.353 

3.943  *** 

4.586 

%  Change  -  R&D  Budget 

-4.6E-05 

-0.002 

0.039  t 

1.380 

%  Change  -  Proc.  Budget 

0.0087 

0.717 

0.022 

1.004 

Unexpected  Inflation 

0.269 

0.912 

0.841  * 

1.878 

Packard 

-0.542 

-0.453 

-0.192 

-0.212 

90s  Reforms 

-2.049  t 

-1.549 

-1.247 

-1.145 

2000  Revision 

-0.777 

-1.160 

-1.365  t 

-1.391 

War 

1.160  t 

1.531 

0.930  t 

1.602 

AR(1) 

0.631  *** 

4.813 

0.500  *** 

3.222 

F-statistic 

9.554  *** 

8.867  *** 

Number  of  Obs 

66 

66 

Number  of  Groups 

3 

3 

R-squared 

0.635 

0.626 

Adjusted  R-squared 

0.568 

0.555 

Durbin-Watson  stat 

1.652 

2.206 

Fixed-Effects 

Yes 

Yes 

Robust  Standard  Errors 

Yes 

Yes 

***statistical  significance  at  0.01  level,  **  0.05  level,  *  0.10  level,  t  0.20  level 
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